acceptodds
Under review as a conference paper at ICLR 2027

On Policy Self Distillation for Learning Kohn-Sham Hamiltonian

Abstract

Neural networks that predict the Kohn–Sham Hamiltonian provide initial guesses that shorten self-consistent field (SCF) calculations. However, every training label is itself a converged SCF calculation. Extending a model to new molecules or functionals therefore requires a new labeled dataset. We observe that a few SCF iterations on the model's own prediction return a corrected Hamiltonian, which indicates how every matrix entry should change. Building on this, we propose on-policy self-distillation with the SCF map (OPSD), which uses the SCF solver as the teacher. A pretrained flow model predicts Hamiltonians, the solver corrects them, and the model learns from the corrections before generating the next round. The model thus amortizes the solver, using only geometries and standard SCF software. In eight transfer settings, including a new functional, implicit solvent, and larger molecules, OPSD has a lower mean error than fine-tuning on converged labels. After a change of functional, for example, the HOMO–LUMO gap error is 2.2 mHa with OPSD and 42 mHa with fine-tuning. On QH9, OPSD lowers the gap error of QHFlow2 models by up to 68%. On the 58-element HELM benchmark, it stabilizes the orbital spectrum of a pretrained QHFlow2 model. OPSD could thus serve as a general method for improving the transferability of Hamiltonian models.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Density Functional Theory, Generative Modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_3x7oOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes On-Policy Self-Distillation with the SCF Map (OPSD), a post-training method for improving Kohn–Sham Hamiltonian prediction without requiring converged SCF labels. The approach uses a pretrained flow-based model to generate Hamiltonian predictions and applies a few SCF iterations to construct corrected training targets. Residual-based acceptance and trust-radius clipping are introduced to filter and constrain these corrections. The model is then updated through a paired distillation loss, a wavefunction alignment loss, and regularization against the pretrained model, with periodic actor updates enabling on-policy learning. The authors also provide theoretical analyses connecting the proposed distillation process to projected gradient descent and comparing matrix-valued feedback with scalar rewards. Experiments on QH9, HELM, MD17, and eight transfer settings evaluate the effectiveness of OPSD.

Critical strengths
  1. The paper provides a useful theoretical analysis connecting SCF correction-based distillation to projected gradient descent on the local DFT energy under specific assumptions. It also discusses how the choice of fitting metric affects convergence.
Critical weaknesses
  1. The assumption that finite-step SCF corrections provide reliable fixed training targets is not sufficiently justified. SCT explicitly emphasizes that the reconstructed Hamiltonian should not be treated as a fixed target, since it is not necessarily closer to the converged solution, and instead requires backpropagation through the reconstruction process to minimize the self-consistency loss. Although OPSD uses multiple SCF iterations and introduces residual-based acceptance, Trust Radius, and a Negative Branch to mitigate this concern, these mechanisms do not guarantee that the corrected Hamiltonian provides a beneficial learning direction. The paper lacks direct empirical evidence validating the reliability of these corrections as supervision targets.
  2. The paper introduces several mechanisms to stabilize training, including residual-based acceptance, Trust Radius, the Negative Branch, and L_base regularization, but does not provide independent ablation studies for any of them. This makes it unclear whether these components are necessary or how much they contribute to the reported improvements.
  3. The comparison with SCT is not sufficiently controlled, as OPSD incorporates an additional Wavefunction Alignment (WA) loss that is absent from the SCT baseline. Since the WA loss directly supervises orbital energies and orbital mixing, it may account for the substantial improvements in HOMO, LUMO, and HOMO–LUMO gap prediction. A comparison between SCT and OPSD with the WA loss removed is needed to determine whether these improvements stem from the proposed on-policy distillation mechanism.
  4. OPSD requires 3–4 SCF iterations and an additional Fock build to construct each correction target, whereas SCT evaluates a single SCF map with backpropagation through the solver. Although OPSD reuses correction targets through a replay buffer, the paper does not report a direct comparison of total Fock builds or wall-clock training costs under matched computational budgets. This makes it difficult to assess the computational efficiency of OPSD relative to SCT.
  5. Theorem 1 establishes convergence only for an idealized SCF iteration under a specific response-weighted quadratic metric, while the actual OPSD implementation uses a different objective involving MAE, scaled MSE, and WA losses. Moreover, Appendix C.7 demonstrates that changing the fitting metric can lead to divergent learned updates even when the SCF teacher is contractive and the residual-based acceptance criterion is satisfied. Therefore, the theoretical analysis does not establish the convergence properties of the actual OPSD algorithm.
Questions for authors
  1. In the unseen-scaffold transfer experiment, supervised fine-tuning (SFT) increases the HOMO–LUMO gap MAE from 0.580 mHa for the pretrained model to 1.648 mHa, despite using converged Hamiltonian labels. What causes this substantial performance degradation? Is it due to overfitting, catastrophic forgetting, or suboptimal optimization settings? The authors should investigate this behavior and clarify whether the reported advantage of OPSD over SFT persists with a properly tuned and regularized SFT baseline.
  2. Why is SCT excluded from all eight transfer experiments, despite being the most directly relevant label-free baseline? Can the authors provide comparisons with SCT under comparable computational budgets, particularly for functional transfer, implicit solvent, and molecular size generalization? A WA-augmented SCT baseline would also help disentangle the contribution of the additional orbital supervision.
  3. How much of the improvement comes specifically from the on-policy training mechanism? The paper does not compare OPSD against an alternative that uses a fixed pretrained actor to generate SCF-corrected pseudo-labels without periodically refreshing the actor. Such a comparison, under matched SCF correction and training budgets, would help establish whether on-policy updates are essential or whether the gains can largely be attributed to learning from SCF-corrected targets.
4 comments

We sincerely thank the reviewer for the thoughtful, constructive review. Your questions and suggested controls helped us significantly strengthen the empirical analysis, ablations, and theoretical discussion in the paper. We ran controlled experiments from the submitted checkpoint and code to address each of the five weaknesses and three questions. The table below summarizes the new results, and the following sections detail each experiment and the corresponding revisions to the manuscript.

SettingMetricControlOPSD
Accepted four-cycle target closer to the converged energy / occupied energies / gap (all benchmarks)% of accepted correctionsN/A100 / 99.9 / 97.1
Fixed pretrained actor (same correction budget, functional transfer)Gap MAE (mHa)11.282.16
Plain regression to refreshed targets (functional transfer)Gap MAE (mHa)15.282.16
OPSD without replay (functional transfer)Gap MAE (mHa)7.832.16
SCT vs. OPSD without WA (no orbital loss on either side, QH9)Gap MAE reduction vs. QHFlow2-s−3.1%−15.0%
SCT [1] (same checkpoint & up to 2,000 updates, functional transfer)Gap MAE (mHa)91.472.16
SCT [1] (same checkpoint & up to 2,000 updates, size transfer)Gap MAE (mHa)7.123.51
WA-SCT (SCT with WA on its own rebuilt Fock matrix, functional transfer)Gap MAE (mHa)7.792.16
WA-SCT (size transfer)Gap MAE (mHa)5.093.51
WA-SCT (QH9, trained to convergence)Gap MAE (μEₕ)201.8147.8
Total compute cost vs. SFT (incl. label generation, functional transfer)GPU-h / CPU-h (32 cores)1.5 / 5442.0 / 288
Training time vs. SCT (QH9, 8 H100, 64 molecules per update)s per update (total GPU-h)4.72 (115)0.64 (48)

W1. Reliability of Few-Step SCF Targets. Figure R2 (anonymous link): https://ibb.co/ns6tGDwY

We thank the reviewer for raising this important point. As insightfully observed in SCT [1], a single undamped Fock rebuild can overcorrect the predicted Hamiltonian: in our evaluation, it moved the occupied orbital energies away from the converged reference in 71% of cases. To avoid this overcorrection, OPSD runs a few DIIS cycles per refinement (four in our experiments) and retains the resulting matrix only if it reduces the SCF residual relative to the model's prediction. We evaluated these targets directly:

  1. Distance to the converged solution (Figure R2a): We evaluated 5,088 refinements generated from checkpoints across QH9, functional transfer, and size transfer. Of these, 5,044 (99.1%) reduced the SCF residual and were accepted, including 2,239 of 2,240 at the final checkpoints where predictions are already accurate. After trust-region clipping, 100% of the accepted targets were closer to the converged reference in total energy and electron density, 99.9% were closer in occupied orbital energies, and 97.1% were closer in the HOMO-LUMO gap.
  2. Gradient alignment (Figure R2b): On the functional-transfer task, we compared the parameter gradients of the OPSD loss (paired loss, orbital-energy term, and pretrained anchor) using fixed four-cycle SCF targets versus fixed converged labels on the same batches and flow states. Across all 320 batches, the cosine similarity was positive in every case, with a mean of 0.93. In contrast, using a single Fock rebuild as the fixed target produced gradients that opposed the converged-label gradient late in training (cosine similarity −0.04 to −0.39).
  3. Learner update (Figure R2c): Starting from the saved training state at update 1,000 of the functional-transfer run, we independently replayed one learner update for each of 64 training batches (8 molecules each, 512 in total), scoring each update on its own batch at the same flow states. The SCF targets have a lower residual than the actor's prediction for every molecule (119 vs. 1,171 μEₕ on average). Relative to the pre-update prediction, the update reduces the learner's residual on 96.7% of molecules (median change −1.1 μEₕ) and yields a lower residual than the actor on 98.2%. This holds in held-out evaluations on 128 molecules as well, where the updated learner outperforms the actor in 98.5% of comparisons (median difference −3.6 μEₕ).

Together, these results suggest that a few DIIS cycles combined with a residual check move targets toward the converged reference and provide reliable update directions throughout training. We will add this empirical analysis and explicitly discuss SCT's observation regarding single-step overcorrection in Section 3.1.

W2 + Q3. Stabilizer Ablations and Fixed Pretrained Actor. Starting from the same checkpoint with a matched correction budget and loss, we ablated each component in turn:

Removed componentFunctional-transfer gap MAE (mHa)Solvent Dipole (D)
None (OPSD)2.160.023
Actor refresh (fixed pretrained actor, Q3)11.280.320
Plain regression to refreshed targets (same WA loss, pretrained anchor, and budget)15.280.245
Replay buffer7.830.053
Trust-region clipping2.900.036
Pretrained anchor (λbase\lambda_{\text{base}})7.940.026
WA distillation15.320.084
Negative branch4.320.027

As the table shows, plain regression to refreshed targets is about 7× worse than OPSD on the gap and 11× worse on the dipole, showing that target refresh alone is not enough. Conversely, keeping the actor frozen at the pretrained model (Q3) increases the error to 11.28 mHa and 0.320 D (and leaves a 10× larger measured SCF residual than OPSD, 19.72 vs. 1.97 mHa) because the corrections are always generated from the frozen pretrained model's predictions. Removing WA distillation, the replay buffer, the anchor, or the negative branch likewise increases error across both tasks. At four DIIS cycles, trust-region clipping gives a smaller improvement and every candidate on QH9 passes the residual check, whereas with a single Fock rebuild the check rejects 67% of updates. We will include this ablation table in the main text.

W3 + Q2. The WA Loss, SCT in Transfer, and SCT with WA. We thank the reviewer for suggesting these controlled comparisons, especially the WA-SCT variant. To separate the effect of the on-policy distillation loop from that of the Wavefunction Alignment (WA) loss [2], we ran both suggested experiments against SCT [1]: removing the WA loss from OPSD, and adding the WA loss to SCT.

  • OPSD without WA vs. SCT on QH9: When we remove LWAL_{\text{WA}} from OPSD so that neither method uses an orbital loss, OPSD reduces the QHFlow2-s gap MAE by 15.0%, compared with 3.1% for SCT. It also achieves larger reductions than SCT on LUMO MAE, at 11.9% versus 1.4%, and on occupied-orbital MAE, at 4.5% versus 1.1%.
  • SCT and WA-SCT across benchmarks: Applying LWAL_{\text{WA}} directly to SCT's single-step rebuilt Fock matrix is a very insightful suggestion. Specifically, this WA-SCT baseline adds LWAL_{\text{WA}}(HθH_{\theta}; F[P(HθH_{\theta})]) to the SCT loss, using the eigenpairs of F[P(HθH_{\theta})] as the WA reference. Because gradients are propagated through F[P(HθH_{\theta})], WA-SCT, like plain SCT, backpropagates through the density matrix, the Kohn-Sham Fock build, and its two-electron analytic response at every update. To investigate this thoroughly, we implemented these backward passes using gradient kernels derived from GPU4PySCF (reaching 4.72 s per update for SCT and 4.9 s for WA-SCT on 8 H100 GPUs), verified them against PySCF and finite differences, and evaluated both plain SCT and WA-SCT from the same checkpoint as OPSD, each run for up to 2,000 updates on the transfer tasks and trained to convergence on QH9:
    • Functional transfer: Plain SCT effectively reduces the SCF residual by 43% even under this cross-functional shift, though the gap MAE remains 91.47 mHa. As the reviewer anticipated, adding WA to SCT dramatically improves transfer, lowering the gap MAE more than tenfold to 7.79 mHa, compared with 2.16 mHa for OPSD (which achieves lower error on 170 of 200 test molecules). While WA-SCT reduces the HOMO-LUMO gap error, its individual HOMO energy error remains large at 35.40 mHa, compared with 0.89 mHa for OPSD.
    • Size transfer and QH9: On the size transfer, the gap MAE is 7.12 mHa for SCT, 5.09 mHa for WA-SCT, and 3.51 mHa for OPSD. On QH9-Stable-ID, WA-SCT reaches a gap MAE of 201.8 μEₕ, compared with 147.8 μEₕ for OPSD.
    • Implicit-solvent transfer: Applying SCT in solvent requires analytic gradients through the solvation Fock build. Our differentiable SCT implementation uses GPU4PySCF kernels, which do not include the ddCOSMO model used in this task, so we did not extend SCT to it for the rebuttal. We see the two approaches as complementary: SCT optimizes the self-consistency loss directly when a differentiable Fock build and its response are available, while OPSD only requires forward SCF evaluations. This is useful when those response terms are not available, for example in production codes like ORCA that do not expose gradients through the Fock build.

We are very grateful to the reviewer for proposing this insightful baseline, which has substantially strengthened our experimental analysis and significantly improved the paper. We will include these comparisons prominently in the main text.

W4. Compute Cost Against SCT. Below we compare the Fock-build count and compute cost of SFT, SCT, OPSD, and RL across QH9:

MetricSFTSCTOPSDRL (Diffusion NFT Group size of 8)
Fock builds per training geometry (QH9)06.83.254.2
Wall time per update on QH9 (8 H100, batch 64)0.38 s4.72 s0.64 s5.32 s
Total training cost on QH942 GPU-h115 GPU-h48 GPU-h130 GPU-h

While SCT evaluates a single Fock build per step, OPSD reuses each correction from the replay buffer across updates (a standard practice in modern RL) and uses fewer Fock builds per training geometry overall (3.2 vs. 6.8, or 0.34 M total builds on QH9). In wall-clock time, moreover, at every update SCT builds a Fock matrix and backpropagates through both the eigensolver and the two-electron Fock response for every molecule in the batch, whereas OPSD runs forward-only SCF workers concurrently with the learner under CUDA MPS without differentiating through the solver. This makes OPSD 2.4× cheaper than SCT in total training cost on QH9 (48 vs. 115 GPU-hours) and 7.4× faster per update. All methods were trained to convergence. We will add this cost breakdown to the main text.

W5. Scope of Theorem 1. We agree that Theorem 1 analyzes an idealized one-step iteration under a response-weighted metric, and that our theoretical results do not establish convergence of the implemented stochastic training procedure. Theorem 4 (Appendix C.9) complements Theorem 1 with a conditional bound for exact projected fitting in a fixed positive-definite quadratic metric: if the finite-budget teacher uniformly contracts the error in that metric, repeated fitting reduces it to a neighborhood set by the model's approximation capacity, and reaches self-consistency in the realizable case. As shown in Appendix C.7 and discussed in W1, a single Fock rebuild can violate this condition, so OPSD uses a short DIIS refinement instead. In practice, measured in our implemented training loss, 5,026 of the 5,044 accepted four-cycle targets (99.6%) are closer to the converged reference than the actor's prediction. This is an empirical check of the theorem's uniform-contraction assumption, and we will make this distinction explicit in Section 4.

Q1. SFT on Unseen Scaffolds. We apologize for not clarifying the SFT training setup on this split. In the submission, to avoid per-benchmark hyperparameter tuning, both SFT and OPSD used the same fixed optimization setup across all transfer tasks (a constant learning rate of 10−510^{-5} with early stopping on validation loss and a pretrained anchor weight of 10−410^{-4}). Early stopping selects the checkpoint on validation molecules from the seen scaffolds, so with a constant learning rate of 10−510^{-5} SFT can overfit to the adaptation scaffolds without this appearing in model selection, which degrades the unseen-scaffold gap. When we sweep the learning rate over {1, 3, 5, 7, 10} × 10−610^{-6} with early stopping for both methods, SFT improves over the pretrained model (0.55 mHa), while OPSD improves further to 0.26 mHa and maintains its advantage across the transfer benchmarks:

EndpointSFT reported → tunedOPSD reported → tuned
Unseen-scaffold gap (mHa)1.65 → 0.550.40 → 0.26
Functional gap (mHa)42.48 → 40.372.16 → 1.99
Solvent dipole (D)0.182 → 0.1690.023 → 0.022

References

  • [1] Zhang, H. et al. Self-Consistency Training for Density-Functional-Theory Hamiltonian Prediction. ICML 2024.
  • [2] Li, Y. et al. Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular Systems. ICLR 2025.
Reviewer_3hKXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes on-policy self-distillation with the SCF map for adapting pretrained Kohn–Sham Hamiltonian models without converged target labels. A frozen actor generates predictions, a standard SCF solver supplies finite-step corrections, and a learner fits accepted corrections using paired matrix losses, wavefunction alignment, and anchoring. The paper provides an idealized convergence analysis and a matrix-versus-scalar feedback complexity comparison. Validation covers QH9, HELM, MD17/rMD17, and eight transfer settings, reporting improved electronic observables and reduced solver costs.

Critical strengths
  • A practical approach to an important adaptation bottleneck. OPSD obtains structured supervision from standard electronic-structure software without differentiating through the solver or requiring converged adaptation labels. This makes the approach potentially useful across functionals and environments where generating new training datasets is expensive.

  • Substantial improvements supported by convincing examples. Functional-transfer gap MAE decreases from 42.48 mHa with supervised fine-tuning to 2.16 mHa with OPSD, with improvements on 196/200 molecules. Solvent-transfer dipole errors improve on 193/200 molecules. These results demonstrate benefits beyond isolated outliers. The functional-transfer budget comparison also reports reaching the final supervised baseline’s error with at least 7.7 times fewer Fock builds, supporting the method’s practical motivation.

Critical weaknesses
  • The main experiments do not sufficiently establish which components produce the gains. OPSD combines on-policy target generation, paired losses, wavefunction alignment, anchoring, replay, and actor averaging. However, the main text lacks comparisons against ordinary regression to periodically refreshed SCF-corrected targets under otherwise comparable conditions. Matching supervised fine-tuning’s learning rate, updates, and wavefunction-alignment loss also does not establish that its stabilization is comparable. This limits attribution of the improvements to the proposed distillation formulation.

  • Improvement of the actual training target and learned update is insufficiently validated. The acceptance criterion checks whether the un-clipped SCF candidate reduces the residual, whereas training uses a subsequently clipped target. For a nonlinear SCF map, residual improvement of the former does not automatically imply improvement of the latter. Furthermore, Theorem 1 assumes quadratic fitting in a response metric that differs from the implemented loss. These acknowledged idealizations leave a gap between the theoretical energy-descent interpretation and the practical update.

Questions for authors

I will raise my rating if the following questions are adressed.

  • How does OPSD compare with ordinary regression to periodically refreshed SCF-corrected targets under the same wavefunction-alignment loss, anchoring, and computational budget? This comparison would clarify whether the paired-loss formulation provides benefits beyond iterative pseudo-label training.

  • Does supervised fine-tuning use comparable pretrained-model anchoring, and how were its optimization settings selected? Would OPSD’s advantage persist when SFT receives the same stabilization and an independently tuned learning rate and stopping point?

  • Were the RL, SCT, and OPSD comparisons conducted under matched computational budgets? Please report total Fock builds and training time for each method, including the overhead of differentiating through the solver for SCT, to establish whether the reported improvements reflect greater computational efficiency.

  • Is residual improvement verified after clipping the SCF correction? Since acceptance evaluates the un-clipped candidate, how frequently do the actual clipped target and subsequent learner prediction improve the residual relative to the actor’s prediction?

4 comments

We sincerely thank the reviewer for the thoughtful, constructive review. Your questions and suggested controls helped us significantly strengthen the empirical analysis, ablations, and theoretical discussion in the paper. We ran the requested controls and evaluations from the submitted checkpoint and code, and the table below summarizes the new results.

SettingMetricControlOPSD
Plain regression to refreshed SCF targets (functional transfer, same correction budget, WA loss, and pretrained anchor)Gap MAE (mHa)15.282.16
Fixed pretrained actor (functional transfer, same correction budget)Gap MAE (mHa)11.282.16
OPSD without replay (functional transfer)Gap MAE (mHa)7.832.16
Total compute cost vs. SFT (incl. label generation, functional transfer)GPU-h / CPU-h (32 cores)1.5 / 5442.0 / 288
Training time vs. SCT / RL (QH9, 8 H100, 64 molecules per update)s per update (total GPU-h)4.72 (115) / 5.32 (130)0.64 (48)
Clipped four-cycle target closer to the converged Hamiltonian / occupied energies / gap (QH9)% of accepted correctionsN/A100 / 100 / 97.2

W1 + Q1. Plain Regression to Refreshed SCF Targets and Component Ablations. We thank the reviewer for suggesting this control. Starting from the same checkpoint with a matched four-cycle correction budget and loss, we compared OPSD against plain regression to periodically refreshed SCF targets (Q1) and individual component ablations:

Removed componentFunctional-transfer gap MAE (mHa)Solvent Dipole (D)
None (OPSD)2.160.023
Plain regression to refreshed targets (Q1; same WA loss, pretrained anchor, and budget)15.280.245
Actor refresh (fixed pretrained actor)11.280.320
Replay buffer7.830.053
Trust-region clipping2.900.036
Pretrained anchor (λbase\lambda_{\text{base}})7.940.026
WA distillation15.320.084
Negative branch4.320.027

As the table shows, plain regression to refreshed targets (Q1) is about 7× worse than OPSD on the gap and 11× worse on the dipole, showing that target refresh alone is not enough. Conversely, keeping the actor frozen at the pretrained model increases the error to 11.28 mHa and 0.320 D (and leaves a 10× larger measured SCF residual than OPSD, 19.72 vs. 1.97 mHa) because the corrections are always generated from the frozen pretrained model's predictions. Removing WA distillation, the replay buffer, the anchor, or the negative branch likewise increases error across both tasks. We will include this ablation table in the main text.

W1 + Q2. SFT Anchoring and Optimization Settings. We apologize for not highlighting this clearly in the submission. Both SFT and OPSD use the same pretrained anchor weight (10−410^{-4}) and optimization schedule (a constant learning rate of 10−510^{-5} with early stopping), providing a matched comparison. Following your suggestion, we also swept the learning rate over {1, 3, 5, 7, 10} × 10−610^{-6} for both methods with early stopping. OPSD's advantage holds:

EndpointSFT reported → tunedOPSD reported → tuned
Functional gap (mHa)42.48 → 40.372.16 → 1.99
Solvent dipole (D)0.182 → 0.1690.023 → 0.022
Unseen-scaffold gap (mHa)1.65 → 0.550.40 → 0.26

Q3. Compute Cost of SFT, SCT, RL, and OPSD. Below we compare the Fock-build count and compute cost of SFT, SCT [1], OPSD, and RL across QH9:

MetricSFTSCTOPSDRL (Diffusion NFT Group size of 8)
Fock builds per training geometry (QH9)06.83.254.2
Wall time per update on QH9 (8 H100, batch 64)0.38 s4.72 s0.64 s5.32 s
Total training cost on QH942 GPU-h115 GPU-h48 GPU-h130 GPU-h

While SCT evaluates one Fock build per step, OPSD reuses each correction multiple times from its replay buffer and uses fewer Fock builds per training geometry overall (3.2 vs. 6.8, or 0.34 M total builds on QH9). In wall-clock time, SCT backpropagates through the eigensolver and two-electron Fock response at every update, whereas OPSD runs forward-only SCF workers concurrently with the learner under CUDA MPS. As a result, OPSD is 2.4× and 2.7× cheaper than SCT and RL in total training cost on QH9 (48 vs. 115 and 130 GPU-hours) and 7.4× and 8.3× faster per update. All methods were trained to convergence. On the functional transfer, end-to-end training with OPSD takes 2.0 GPU-hours and 288 CPU-hours (32 cores), against 1.5 GPU-hours and 544 CPU-hours for SFT once converged label generation is included. We will add this cost breakdown to the main text.

W2 + Q4. Reliability of Clipped Few-Step SCF Targets. Figure R2 (anonymous link): https://ibb.co/ns6tGDwY

We thank the reviewer for raising this important point. Because the residual check is evaluated on the unclipped candidate and the Kohn-Sham map is nonlinear, we verified both the trust-clipped targets on QH9 and the four-cycle DIIS targets across benchmarks directly against held-out converged DFT labels:

  • Trust-clipped targets on QH9 (Q4): On 1,024 held-out QH9-Stable-ID molecules, 100% of four-cycle candidates pass the residual check and trust-region clipping shortens 12.1% (124/1,024) of the updates. Across all 1,024 clipped targets, 100% have a lower SCF residual than the model's prediction (dropping the mean RMS residual from 32.8 to 4.1 μEₕ), 100% move closer to the converged reference in both Hamiltonian error (median distance ratio 0.035) and occupied orbital energies, and 97.2% move closer in the HOMO-LUMO gap (98.4% on the 124 actively clipped molecules).
  • Learner after one update (Q4; Figure R2c): Starting from the saved training state at update 1,000 of the functional-transfer run, we independently replayed one learner update for each of 64 training batches (8 molecules each, 512 in total), scoring each update on its own batch at the same flow states. The SCF targets have a lower residual than the actor's prediction for every molecule (119 vs. 1,171 μEₕ on average). Relative to the pre-update prediction, the update reduces the learner's residual on 96.7% of molecules (median change −1.1 μEₕ) and yields a lower residual than the actor on 98.2%. This holds in held-out evaluations on 128 molecules as well, where the updated learner outperforms the actor in 98.5% of comparisons (median difference −3.6 μEₕ).
  • Four-cycle DIIS targets across benchmarks (Figure R2a): Across 5,088 refinements from checkpoints on QH9, functional transfer, and size transfer, 5,044 (99.1%) reduced the SCF residual and were accepted. After trust-region clipping, 100% of the accepted targets were closer to the converged reference in total energy and electron density, 99.9% in occupied orbital energies, and 97.1% in the HOMO-LUMO gap. By contrast, a single Fock rebuild moved occupied orbital energies away from the reference in 71% of cases.
  • Gradient alignment (Figure R2b): On the functional-transfer task, we compared the parameter gradients of the OPSD loss (paired loss, orbital-energy term, and pretrained anchor) using fixed four-cycle SCF targets versus fixed converged labels on the same batches and flow states. Across all 320 batches, the cosine similarity was positive in every case, with a mean of 0.93. In contrast, using a single Fock rebuild as the fixed target produced gradients that opposed the converged-label gradient late in training (cosine similarity −0.04 to −0.39).

Together, these results suggest that a few DIIS cycles with the residual check and trust-region clipping consistently move targets toward the converged reference throughout training. We will add this analysis to Appendix C.12 and Section 3.1.

W2 (continued). Scope of Theorem 1. We appreciate the reviewer's careful reading of the theory. We agree that Theorem 1 analyzes an idealized one-step iteration under a response-weighted metric, and that our theoretical results do not establish convergence of the implemented stochastic training procedure. We apologize for not highlighting this more clearly in the manuscript. Theorem 4 (Appendix C.9) complements Theorem 1 with a conditional bound for exact projected fitting in a fixed positive-definite quadratic metric: if the finite-budget teacher uniformly contracts the error in that metric, repeated fitting reduces it to a neighborhood set by the model's approximation capacity, and reaches self-consistency in the realizable case. Because a single Fock rebuild can violate this condition (Appendix C.7), OPSD uses a short DIIS refinement instead. In practice, measured in our implemented training loss, 5,026 of the 5,044 accepted four-cycle targets (99.6%) are closer to the converged reference than the actor's prediction. This is an empirical check of the theorem's uniform-contraction assumption, and we will make this distinction explicit in Section 4.

We hope these results and clarifications address your questions. We are grateful for your constructive feedback, which has substantially improved the paper, and we would be glad to provide any further analysis.

References

  • [1] Zhang, H. et al. Self-Consistency Training for Density-Functional-Theory Hamiltonian Prediction. ICML 2024.
Reviewer_3hKXOpenReview ↗

After rebuttal

I appreciate the response and clarification. The authors’ additional experiments and clarifications address my main concerns about the contribution of OPD and the reliability of the clipped SCF targets, so I am raising my score.

Reviewer_B1yeOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This work brings an RL-inspired perspective to Kohn–Sham Hamiltonian prediction. It introduces OPSD, which uses a DFT solver to refine a model’s Hamiltonian predictions through a few SCF steps and then uses the corrected matrices as training targets. With QHFlow2 as the backbone, the authors report modest gains in some in-distribution metrics and better results than supervised fine-tuning on the primary metrics across eight out-of-distribution settings. I find it interesting that the work treats pretrained Hamiltonian predictors as a foundation for further adaptation and brings ideas from on-policy learning in language and vision models into electronic-structure prediction through DFT-based feedback.

Critical strengths

The connection to learning methods from other fields offers an interesting perspective. Using a DFT solver to correct model predictions and then training on those corrections has been explored in prior work. Nevertheless, I find it interesting to treat QHFlow2 as a foundation model and interpret its post-training through ideas from RL and on-policy distillation in language modeling and robotics.

The transfer experiments fit naturally with the paper’s goal. The authors evaluate adaptation in eight settings involving changes in the functional, solvent, geometry, and molecular size. Comparing against supervised fine-tuning from the same checkpoint with the main training conditions matched is also a positive aspect. These experiments align well with the goal of adapting pretrained Hamiltonian models without converged labels for the new conditions.

Critical weaknesses

The overall computational benefit is not established. The introduction identifies the high cost of generating converged DFT labels as a central motivation (lines 43–48). Given this motivation, it is important to show that the gains justify repeatedly calling an expensive DFT solver during training. Figure 4a compares the Fock builds needed for target generation, but the paper does not report the total time and resource usage accounting for model rollouts, repeated SCF corrections, and learner updates. Some gap improvements in Table 1 are substantial, but Hamiltonian MAE does not improve consistently, and some additional energy/force improvements in Table 3 are modest. Without this cost information, it is difficult to judge the practical value of the gains in Tables 1 and 3.

For the standard benchmarks where converged training targets are already available, an important comparison is continued supervised training from the same checkpoint under the same total compute budget. The theoretical analysis explains energy-error minimization within the model’s representable prediction space under idealized assumptions, but it does not establish that the implemented method is more effective than reusing existing labels for further training at the same cost. Allocating the same solver budget to generating additional training data at perturbed geometries would also be a useful comparison.

The paper should also discuss how the cost of repeated solver calls scales with molecular and basis size. The current results do not establish whether OPSD would actually be cheaper than generating converged labels and training on them at the scale of PubChemQH or ∇²DFT, which are cited as motivating examples.

The contributions of the on-policy components and WA distillation remain unclear. The relationship to prior methods that use solver information during training, including SCT and SAIL, should be explained more clearly. SAIL is not cited in the submitted manuscript. Although the RL interpretation is interesting, the current results do not make clear what actor updates, replay, and anchoring each add beyond learning targets obtained from a few SCF steps. The improved gap accuracy over SCT in Table 1 does not by itself establish that the gain comes from the on-policy components. Comparing fixed and refreshed actors under the same correction budget and loss, alongside ablations of the other components, would help clarify their effects.

The role of WA distillation in the paper’s central message is also unclear. In particular, the large improvements from projected WA on OMol are interesting, but it is difficult to separate the benefits of on-policy learning from those of loss design and numerical stabilization.

Questions for authors

Compute budget. On QH9 and MD17/rMD17, how does OPSD compare with continued supervised training on existing converged labels, including a baseline using WA, from the same checkpoint under the same total compute budget?

On-policy contribution. What controlled comparisons isolate the benefits of actor updates, replay, and anchoring from simply fitting targets corrected by a few SCF steps?

OMol controls. Please compare OPSD with continued training on existing converged labels using WA/projected WA, starting from the same checkpoint with matched training data and budgets. In addition to Table 2’s common 10−310^{-3} prediction cutoff, how do the same checkpoints perform with overlap regularization consistent with the reference calculations? Please also report post-training Hamiltonian MAE, if available.


Suggestions

Clarify the relationship to SAIL. I would treat SAIL as concurrent work and therefore do not request a direct experimental comparison. However, it was first posted on arXiv on April 23, 2026, and given its public availability, I believe it could reasonably have been considered and discussed as related work. Please cite SAIL and clarify how its learning objectives and use of the SCF solver differ from those of this paper. (Eberhard et al., Transferable SCF-Acceleration through Solver-Aligned Initialization Learning, 2026)

Distinguish model names from dataset names. HELM is the name of a model, so expressions such as “Post-training on HELM” can misleadingly suggest that it is a dataset. Please explicitly name OMol_CSH_58k as the pretraining dataset and OMol_common_1k and OMol_all_5k as the evaluation sets, and make clear that QHFlow2 is the model being post-trained.

Use consistent energy units. Table 1 reports Hamiltonian and orbital-energy errors in μEh, whereas Table 2 reports orbital-energy errors in eV. This makes it difficult to compare the magnitudes of the errors intuitively across the tables.

10 comments

We sincerely thank the reviewer for the careful and constructive review. Your suggested controls helped us improve the comparisons with supervised training and SCT. We ran these controls from the submitted checkpoint and code. The table below summarizes the new results, and the following sections detail each experiment and the corresponding changes to the manuscript.

SettingMetricBaselineOPSD
Total compute cost vs. SFT (incl. label generation, functional transfer)GPU-h / CPU-h (32 cores)1.5 / 5442.0 / 288
Continued supervised training with WA (same GPU-hours, QH9)Gap MAE (μEₕ)278.2147.8
SFT on new DFT labels at perturbed geometries (larger Fock-build budget, QH9)Gap MAE (μEₕ)316.9147.8
Continued supervised training (all 6 molecules, MD17 / rMD17, QHFlow2-s / QHFlow2-m)Mean Force MAE (meV/Å)1.00 / 0.930.92 / 0.88
Fixed pretrained actor (same correction budget, functional transfer)Gap MAE (mHa)11.282.16
SCT [1] (same checkpoint & up to 2,000 updates, functional transfer)Gap MAE (mHa)91.472.16
SCT [1] (same checkpoint & up to 2,000 updates, size transfer)Gap MAE (mHa)7.123.51

W1. End-to-End Computational Cost and Practical Trade-offs. Thank you for raising this important point, and we apologize for not reporting the end-to-end cost in the submission. We agree that Figure 4a should be paired with total training times, since target-generation Fock builds alone do not include rollouts or learner updates. On the functional transfer benchmark, we timed both OPSD and SFT end to end, including model rollouts, CPU SCF corrections on 32 cores, and GPU learner updates:

  • OPSD (total): 2.0 GPU-hours and 288 CPU-hours.
  • SFT (including converged label generation): 1.5 GPU-hours and 544 CPU-hours.

OPSD adds 0.5 GPU-hours over SFT and uses about half the CPU-hours (288 vs. 544), with the GPU learner updates and the CPU SCF corrections running concurrently in Ray. On QH9, where the SCF workers share the learner's GPUs under CUDA MPS, the table below gives the Fock-build count and end-to-end compute cost of SFT, SCT [1], OPSD, and RL. All methods were trained to convergence.

MetricSFTSCTOPSDRL (Diffusion NFT Group size of 8)
Fock builds per training geometry (QH9)06.83.254.2
Wall time per update on QH9 (8 H100, batch 64)0.38 s4.72 s0.64 s5.32 s
Total training cost on QH942 GPU-h115 GPU-h48 GPU-h130 GPU-h

On the four QH9 tasks, OPSD reduces the gap MAE of QHFlow2-s by 27% to 64.1%, compared with 1.5% to 5.2% for RL and SCT [1]. On QH9, OPSD post-training takes 48 GPU-hours and 0.34 M Fock builds, against 115 and 130 GPU-hours for SCT and RL. OPSD adds only 6 GPU-hours over SFT (48 vs. 42 GPU-hours). We will report these end-to-end compute costs in Section 5 and Appendix E.

W2 + Q1. Same-Budget Continued Supervised Training. This is an important control, and we thank the reviewer for proposing it. We ran continued supervised training from the same pretrained checkpoints on the existing converged labels at matched compute budgets across three benchmarks:

  • QH9: Both methods start from QHFlow2-s with the same loss (LHamL_{\text{Ham}} + LWAL_{\text{WA}} + LanchorL_{\text{anchor}}) and GPU-hour budget.
  • MD17 and rMD17: Values are averaged over all six molecules on both benchmarks, and OPSD gives per-molecule reductions of up to 18.7% on salicylic acid and 9.2% on aspirin.
  • OMol (OMol_common_1k): Both methods start from the same checkpoint and use the same 64 training molecules with projected WA (see W5 + Q3).

At matched compute budgets, OPSD gives lower errors than continued supervised training on every metric:

BenchmarkMetricContinued SFTOPSD
QH9 (QHFlow2-s)Gap MAE (μEₕ)278.2147.8
MD17 and rMD17 (QHFlow2-s)Mean force MAE (meV/Å)1.000.92
MD17 and rMD17 (QHFlow2-m)Mean force MAE (meV/Å)0.930.88
OMol_common_1kHamiltonian MAE (μEₕ)592.1528.2
OMol_common_1kOccupied-orbital MAE (eV)25.648.34
OMol_common_1kGap MAE (eV)1.4921.471

We will add these comparisons to Section 5 and Appendix E.

W2 + Q1 (continued). Supervised Training on New Perturbed QH9 Geometries. We also tested whether spending a larger Fock-build budget on new converged DFT labels at perturbed geometries matches OPSD. We added Gaussian coordinate noise (σ = 0.02 Å) to a random subset of 52,332 QH9-Stable-ID training molecules and computed converged B3LYP/def2-SVP labels. Generating these 52,332 labels takes 0.63 M Fock builds, compared with 0.34 M builds for OPSD, and we will make this dataset available upon publication. We then trained QHFlow2-s on these labels with the same Hamiltonian + WA loss and schedule:

ModelHOMO (μEₕ)LUMO (μEₕ)Gap (μEₕ)
QHFlow2-s (starting checkpoint)148.4304.7411.4
SFT on 52,332 new converged labels at perturbed geometries185.8268.6316.9
OPSD84.8120.5147.8

Training on the new perturbed-geometry labels reduces the gap MAE from 411.4 to 316.9 μEₕ, while the HOMO MAE increases from 148.4 to 185.8 μEₕ, possibly because the small QHFlow2-s model lacks the capacity to fit the new geometries (we will include results with a larger model in the final version); OPSD reaches 147.8 μEₕ on the gap and 84.8 μEₕ on the HOMO. Because OPSD does not require pre-converged labels, it can also be run directly on newly sampled geometries. We will include this control in Appendix E.

W3. Cost Scaling with Molecular and Basis Set Size. Figure R1 (anonymous link): https://ibb.co/VpPD329B

Thank you for this question, which prompted a more complete cost analysis. We agree that Fock-build counts alone do not capture how solver cost scales with molecular and basis-set size. We therefore added a new analysis to Figure R1 that reports two quantities: the number of Fock builds needed to generate a converged label, and the difference in solver time between generating a converged label and performing one OPSD correction (assuming four DIIS cycles per correction). We examine how both vary with molecular size, basis set, and HOMO-LUMO gap. The Fock-build counts for molecular size and gap come from our internal dataset. This dataset contains over 600,000 molecules with 2 to 350 atoms, computed at the ωB97X-D3(BJ)/def2-TZVPD level. The timing comparisons use subsets of this dataset:

  • System size (Figure R1a,d): The number of DIIS iterations to convergence grows slowly with system size, scaling as Natoms0.06N_{\text{atoms}}^{0.06} from 2 to 350 atoms within each molecule type, with medians of 14 to 24 Fock builds across molecule types. However, because each Fock build becomes much more expensive on larger systems, similar iteration counts can correspond to very different computational costs. Timing a subset of these molecules (n = 1,506) confirmed that the time difference between generating a converged label and performing a single correction increases substantially with molecular size.
  • Basis-set size (Figure R1b,e): We also ran SCF and OPSD on a subset of molecules (n = 147) across SVP, TZVP, TZVPD, and QZVPP. Convergence requires roughly 14 Fock builds regardless of basis set. However, each build becomes significantly more expensive as the basis grows. As a result, skipping the remaining SCF iterations saves more time with larger basis sets.
  • HOMO-LUMO gap (Figure R1c,f): Beyond system size and basis set, we discover that the HOMO-LUMO gap (Δεgap\Delta\varepsilon_{\text{gap}}) is another key factor governing timing. For closed-shell, metal-free molecules, convergence slows as the gap narrows. The number of Fock builds rises from about 13 for wide-gap systems (>10 eV) to about 30 at 1–2 eV. This follows an empirical power law of Δεgap−0.48\Delta\varepsilon_{\text{gap}}^{-0.48}. This aligns with SCF convergence theory: a smaller gap worsens the condition number of the electronic Hessian as κ ∝ Δεgap−1\Delta\varepsilon_{\text{gap}}^{-1} [4]. Because DIIS/Anderson acceleration acts as a Krylov method related to GMRES [5, 6], its iteration count scales under standard assumptions as O(√κ) ∝ Δεgap−0.5\Delta\varepsilon_{\text{gap}}^{-0.5} [7], closely matching our observed −0.48 exponent. Timing a subset of 30–60 atom molecules (n = 345) confirms that the resulting wall-clock savings are correspondingly larger for narrow-gap systems.

W3 (continued). These measurements compare single target-generation operations. Over a full adaptation run, OPSD uses V(b+1) Fock builds per training geometry, where V is the average number of newly generated corrections per geometry and b is the number of DIIS cycles per correction. We note that reusing a correction from the replay buffer requires no additional Fock builds. In our runs, OPSD uses 3.2 Fock builds per training geometry on QH9 (0.34 M in total) and 4.9 on the functional transfer.

We report the corresponding end-to-end compute costs, including model rollouts and learner updates, in our response to W1, and will include Figure R1 and the timing protocols in the Appendix.

W4 + Q2. Isolating On-Policy Components, Extended SCT Comparisons, and Relation to SAIL. We appreciate this suggestion. We agree that the QH9 comparison with SCT does not separate the parts of the on-policy loop. On the functional and implicit-solvent transfer tasks, holding the correction budget and loss fixed, we removed each component in turn:

Removed componentFunctional-transfer gap MAE (mHa)Solvent Dipole (D)
None (OPSD)2.160.023
Actor refresh (fixed pretrained actor, Q2)11.280.320
Plain regression to refreshed targets (same WA loss, pretrained anchor, and budget)15.280.245
Replay buffer7.830.053
Trust-region clipping2.900.036
Pretrained anchor (λbase\lambda_{\text{base}})7.940.026
WA distillation15.320.084
Negative branch4.320.027

As the table shows, keeping the actor frozen at the pretrained model (Q2) restricts training to targets corrected by a few SCF steps from fixed predictions. This increases the error to 11.28 mHa and 0.320 D, yielding a 6.6× larger Hamiltonian MAE and a 10× larger SCF residual than OPSD (19.72 vs. 1.97 mHa). Meanwhile, plain regression to refreshed targets is about 7× worse than OPSD on the gap and 11× worse on the dipole, showing that target refresh alone is not enough. Removing the replay buffer, pretrained anchor, WA distillation, or negative branch similarly increases error across both tasks.

SCT on the transfer benchmarks: We also ran SCT [1] on the functional and size transfers from the same starting checkpoint with OPSD's schedule (up to 2,000 updates with early stopping, same learning rate and batch size) and exact Coulomb and exchange matrices at the target level of theory:

  • Functional transfer (Gap MAE): From a starting error of 155.39 mHa, SCT reaches 91.47 mHa and OPSD reaches 2.16 mHa (lower error on all 200 test molecules). SCT reduces the SCF residual by 43%, while its HOMO MAE is 63.00 mHa compared with 0.89 mHa for OPSD.
  • Size transfer (Gap MAE): SCT reaches 7.12 mHa and OPSD reaches 3.51 mHa (lower error on 46 of 64 test molecules).

Relation to SAIL [2]: Thank you for this suggestion. SAIL is closely related to our work and follows a similar setup. SAIL is initially trained on converged labels to predict corrections to an initial Fock or density matrix guess, similar to QHFlow2. In post-training, SAIL rolls out the model's prediction through a differentiable SCF solver for about 10 DIIS steps and penalizes the occupied-virtual block of the Fock matrix at each step. This block is the part of the self-consistency residual that updates the density, whereas SCT [1] penalizes the full residual. SAIL is thus a natural and elegant extension of SCT: by supervising the solver trajectory across multiple steps, it aligns training with how the prediction is used downstream. The main difference from OPSD is how the solver is used: SCT and SAIL both backpropagate through the solver, while OPSD only runs it forward and fits the predictor to a multiple-cycle DIIS target. We will cite SAIL and add this discussion to the related work. We will include the component ablations and the transfer SCT results in the main text.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.