- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes On-Policy Self-Distillation with the SCF Map (OPSD), a post-training method for improving Kohn–Sham Hamiltonian prediction without requiring converged SCF labels. The approach uses a pretrained flow-based model to generate Hamiltonian predictions and applies a few SCF iterations to construct corrected training targets. Residual-based acceptance and trust-radius clipping are introduced to filter and constrain these corrections. The model is then updated through a paired distillation loss, a wavefunction alignment loss, and regularization against the pretrained model, with periodic actor updates enabling on-policy learning. The authors also provide theoretical analyses connecting the proposed distillation process to projected gradient descent and comparing matrix-valued feedback with scalar rewards. Experiments on QH9, HELM, MD17, and eight transfer settings evaluate the effectiveness of OPSD.
Critical strengths
- The paper provides a useful theoretical analysis connecting SCF correction-based distillation to projected gradient descent on the local DFT energy under specific assumptions. It also discusses how the choice of fitting metric affects convergence.
Critical weaknesses
- The assumption that finite-step SCF corrections provide reliable fixed training targets is not sufficiently justified. SCT explicitly emphasizes that the reconstructed Hamiltonian should not be treated as a fixed target, since it is not necessarily closer to the converged solution, and instead requires backpropagation through the reconstruction process to minimize the self-consistency loss. Although OPSD uses multiple SCF iterations and introduces residual-based acceptance, Trust Radius, and a Negative Branch to mitigate this concern, these mechanisms do not guarantee that the corrected Hamiltonian provides a beneficial learning direction. The paper lacks direct empirical evidence validating the reliability of these corrections as supervision targets.
- The paper introduces several mechanisms to stabilize training, including residual-based acceptance, Trust Radius, the Negative Branch, and L_base regularization, but does not provide independent ablation studies for any of them. This makes it unclear whether these components are necessary or how much they contribute to the reported improvements.
- The comparison with SCT is not sufficiently controlled, as OPSD incorporates an additional Wavefunction Alignment (WA) loss that is absent from the SCT baseline. Since the WA loss directly supervises orbital energies and orbital mixing, it may account for the substantial improvements in HOMO, LUMO, and HOMO–LUMO gap prediction. A comparison between SCT and OPSD with the WA loss removed is needed to determine whether these improvements stem from the proposed on-policy distillation mechanism.
- OPSD requires 3–4 SCF iterations and an additional Fock build to construct each correction target, whereas SCT evaluates a single SCF map with backpropagation through the solver. Although OPSD reuses correction targets through a replay buffer, the paper does not report a direct comparison of total Fock builds or wall-clock training costs under matched computational budgets. This makes it difficult to assess the computational efficiency of OPSD relative to SCT.
- Theorem 1 establishes convergence only for an idealized SCF iteration under a specific response-weighted quadratic metric, while the actual OPSD implementation uses a different objective involving MAE, scaled MSE, and WA losses. Moreover, Appendix C.7 demonstrates that changing the fitting metric can lead to divergent learned updates even when the SCF teacher is contractive and the residual-based acceptance criterion is satisfied. Therefore, the theoretical analysis does not establish the convergence properties of the actual OPSD algorithm.
Questions for authors
- In the unseen-scaffold transfer experiment, supervised fine-tuning (SFT) increases the HOMO–LUMO gap MAE from 0.580 mHa for the pretrained model to 1.648 mHa, despite using converged Hamiltonian labels. What causes this substantial performance degradation? Is it due to overfitting, catastrophic forgetting, or suboptimal optimization settings? The authors should investigate this behavior and clarify whether the reported advantage of OPSD over SFT persists with a properly tuned and regularized SFT baseline.
- Why is SCT excluded from all eight transfer experiments, despite being the most directly relevant label-free baseline? Can the authors provide comparisons with SCT under comparable computational budgets, particularly for functional transfer, implicit solvent, and molecular size generalization? A WA-augmented SCT baseline would also help disentangle the contribution of the additional orbital supervision.
- How much of the improvement comes specifically from the on-policy training mechanism? The paper does not compare OPSD against an alternative that uses a fixed pretrained actor to generate SCF-corrected pseudo-labels without periodically refreshing the actor. Such a comparison, under matched SCF correction and training budgets, would help establish whether on-policy updates are essential or whether the gains can largely be attributed to learning from SCF-corrected targets.
4 comments
W1. Reliability of Few-Step SCF Targets. Figure R2 (anonymous link): https://ibb.co/ns6tGDwY
We thank the reviewer for raising this important point. As insightfully observed in SCT [1], a single undamped Fock rebuild can overcorrect the predicted Hamiltonian: in our evaluation, it moved the occupied orbital energies away from the converged reference in 71% of cases. To avoid this overcorrection, OPSD runs a few DIIS cycles per refinement (four in our experiments) and retains the resulting matrix only if it reduces the SCF residual relative to the model's prediction. We evaluated these targets directly:
- Distance to the converged solution (Figure R2a): We evaluated 5,088 refinements generated from checkpoints across QH9, functional transfer, and size transfer. Of these, 5,044 (99.1%) reduced the SCF residual and were accepted, including 2,239 of 2,240 at the final checkpoints where predictions are already accurate. After trust-region clipping, 100% of the accepted targets were closer to the converged reference in total energy and electron density, 99.9% were closer in occupied orbital energies, and 97.1% were closer in the HOMO-LUMO gap.
- Gradient alignment (Figure R2b): On the functional-transfer task, we compared the parameter gradients of the OPSD loss (paired loss, orbital-energy term, and pretrained anchor) using fixed four-cycle SCF targets versus fixed converged labels on the same batches and flow states. Across all 320 batches, the cosine similarity was positive in every case, with a mean of 0.93. In contrast, using a single Fock rebuild as the fixed target produced gradients that opposed the converged-label gradient late in training (cosine similarity −0.04 to −0.39).
- Learner update (Figure R2c): Starting from the saved training state at update 1,000 of the functional-transfer run, we independently replayed one learner update for each of 64 training batches (8 molecules each, 512 in total), scoring each update on its own batch at the same flow states. The SCF targets have a lower residual than the actor's prediction for every molecule (119 vs. 1,171 μEₕ on average). Relative to the pre-update prediction, the update reduces the learner's residual on 96.7% of molecules (median change −1.1 μEₕ) and yields a lower residual than the actor on 98.2%. This holds in held-out evaluations on 128 molecules as well, where the updated learner outperforms the actor in 98.5% of comparisons (median difference −3.6 μEₕ).
Together, these results suggest that a few DIIS cycles combined with a residual check move targets toward the converged reference and provide reliable update directions throughout training. We will add this empirical analysis and explicitly discuss SCT's observation regarding single-step overcorrection in Section 3.1.
W2 + Q3. Stabilizer Ablations and Fixed Pretrained Actor. Starting from the same checkpoint with a matched correction budget and loss, we ablated each component in turn:
| Removed component | Functional-transfer gap MAE (mHa) | Solvent Dipole (D) |
|---|---|---|
| None (OPSD) | 2.16 | 0.023 |
| Actor refresh (fixed pretrained actor, Q3) | 11.28 | 0.320 |
| Plain regression to refreshed targets (same WA loss, pretrained anchor, and budget) | 15.28 | 0.245 |
| Replay buffer | 7.83 | 0.053 |
| Trust-region clipping | 2.90 | 0.036 |
| Pretrained anchor () | 7.94 | 0.026 |
| WA distillation | 15.32 | 0.084 |
| Negative branch | 4.32 | 0.027 |
As the table shows, plain regression to refreshed targets is about 7× worse than OPSD on the gap and 11× worse on the dipole, showing that target refresh alone is not enough. Conversely, keeping the actor frozen at the pretrained model (Q3) increases the error to 11.28 mHa and 0.320 D (and leaves a 10× larger measured SCF residual than OPSD, 19.72 vs. 1.97 mHa) because the corrections are always generated from the frozen pretrained model's predictions. Removing WA distillation, the replay buffer, the anchor, or the negative branch likewise increases error across both tasks. At four DIIS cycles, trust-region clipping gives a smaller improvement and every candidate on QH9 passes the residual check, whereas with a single Fock rebuild the check rejects 67% of updates. We will include this ablation table in the main text.
W3 + Q2. The WA Loss, SCT in Transfer, and SCT with WA. We thank the reviewer for suggesting these controlled comparisons, especially the WA-SCT variant. To separate the effect of the on-policy distillation loop from that of the Wavefunction Alignment (WA) loss [2], we ran both suggested experiments against SCT [1]: removing the WA loss from OPSD, and adding the WA loss to SCT.
- OPSD without WA vs. SCT on QH9: When we remove from OPSD so that neither method uses an orbital loss, OPSD reduces the QHFlow2-s gap MAE by 15.0%, compared with 3.1% for SCT. It also achieves larger reductions than SCT on LUMO MAE, at 11.9% versus 1.4%, and on occupied-orbital MAE, at 4.5% versus 1.1%.
- SCT and WA-SCT across benchmarks: Applying directly to SCT's single-step rebuilt Fock matrix is a very insightful suggestion. Specifically, this WA-SCT baseline adds (; F[P()]) to the SCT loss, using the eigenpairs of F[P()] as the WA reference. Because gradients are propagated through F[P()], WA-SCT, like plain SCT, backpropagates through the density matrix, the Kohn-Sham Fock build, and its two-electron analytic response at every update. To investigate this thoroughly, we implemented these backward passes using gradient kernels derived from GPU4PySCF (reaching 4.72 s per update for SCT and 4.9 s for WA-SCT on 8 H100 GPUs), verified them against PySCF and finite differences, and evaluated both plain SCT and WA-SCT from the same checkpoint as OPSD, each run for up to 2,000 updates on the transfer tasks and trained to convergence on QH9:
- Functional transfer: Plain SCT effectively reduces the SCF residual by 43% even under this cross-functional shift, though the gap MAE remains 91.47 mHa. As the reviewer anticipated, adding WA to SCT dramatically improves transfer, lowering the gap MAE more than tenfold to 7.79 mHa, compared with 2.16 mHa for OPSD (which achieves lower error on 170 of 200 test molecules). While WA-SCT reduces the HOMO-LUMO gap error, its individual HOMO energy error remains large at 35.40 mHa, compared with 0.89 mHa for OPSD.
- Size transfer and QH9: On the size transfer, the gap MAE is 7.12 mHa for SCT, 5.09 mHa for WA-SCT, and 3.51 mHa for OPSD. On QH9-Stable-ID, WA-SCT reaches a gap MAE of 201.8 μEₕ, compared with 147.8 μEₕ for OPSD.
- Implicit-solvent transfer: Applying SCT in solvent requires analytic gradients through the solvation Fock build. Our differentiable SCT implementation uses GPU4PySCF kernels, which do not include the ddCOSMO model used in this task, so we did not extend SCT to it for the rebuttal. We see the two approaches as complementary: SCT optimizes the self-consistency loss directly when a differentiable Fock build and its response are available, while OPSD only requires forward SCF evaluations. This is useful when those response terms are not available, for example in production codes like ORCA that do not expose gradients through the Fock build.
We are very grateful to the reviewer for proposing this insightful baseline, which has substantially strengthened our experimental analysis and significantly improved the paper. We will include these comparisons prominently in the main text.
W4. Compute Cost Against SCT. Below we compare the Fock-build count and compute cost of SFT, SCT, OPSD, and RL across QH9:
| Metric | SFT | SCT | OPSD | RL (Diffusion NFT Group size of 8) |
|---|---|---|---|---|
| Fock builds per training geometry (QH9) | 0 | 6.8 | 3.2 | 54.2 |
| Wall time per update on QH9 (8 H100, batch 64) | 0.38 s | 4.72 s | 0.64 s | 5.32 s |
| Total training cost on QH9 | 42 GPU-h | 115 GPU-h | 48 GPU-h | 130 GPU-h |
While SCT evaluates a single Fock build per step, OPSD reuses each correction from the replay buffer across updates (a standard practice in modern RL) and uses fewer Fock builds per training geometry overall (3.2 vs. 6.8, or 0.34 M total builds on QH9). In wall-clock time, moreover, at every update SCT builds a Fock matrix and backpropagates through both the eigensolver and the two-electron Fock response for every molecule in the batch, whereas OPSD runs forward-only SCF workers concurrently with the learner under CUDA MPS without differentiating through the solver. This makes OPSD 2.4× cheaper than SCT in total training cost on QH9 (48 vs. 115 GPU-hours) and 7.4× faster per update. All methods were trained to convergence. We will add this cost breakdown to the main text.
W5. Scope of Theorem 1. We agree that Theorem 1 analyzes an idealized one-step iteration under a response-weighted metric, and that our theoretical results do not establish convergence of the implemented stochastic training procedure. Theorem 4 (Appendix C.9) complements Theorem 1 with a conditional bound for exact projected fitting in a fixed positive-definite quadratic metric: if the finite-budget teacher uniformly contracts the error in that metric, repeated fitting reduces it to a neighborhood set by the model's approximation capacity, and reaches self-consistency in the realizable case. As shown in Appendix C.7 and discussed in W1, a single Fock rebuild can violate this condition, so OPSD uses a short DIIS refinement instead. In practice, measured in our implemented training loss, 5,026 of the 5,044 accepted four-cycle targets (99.6%) are closer to the converged reference than the actor's prediction. This is an empirical check of the theorem's uniform-contraction assumption, and we will make this distinction explicit in Section 4.
Q1. SFT on Unseen Scaffolds. We apologize for not clarifying the SFT training setup on this split. In the submission, to avoid per-benchmark hyperparameter tuning, both SFT and OPSD used the same fixed optimization setup across all transfer tasks (a constant learning rate of with early stopping on validation loss and a pretrained anchor weight of ). Early stopping selects the checkpoint on validation molecules from the seen scaffolds, so with a constant learning rate of SFT can overfit to the adaptation scaffolds without this appearing in model selection, which degrades the unseen-scaffold gap. When we sweep the learning rate over {1, 3, 5, 7, 10} × with early stopping for both methods, SFT improves over the pretrained model (0.55 mHa), while OPSD improves further to 0.26 mHa and maintains its advantage across the transfer benchmarks:
| Endpoint | SFT reported → tuned | OPSD reported → tuned |
|---|---|---|
| Unseen-scaffold gap (mHa) | 1.65 → 0.55 | 0.40 → 0.26 |
| Functional gap (mHa) | 42.48 → 40.37 | 2.16 → 1.99 |
| Solvent dipole (D) | 0.182 → 0.169 | 0.023 → 0.022 |
References
- [1] Zhang, H. et al. Self-Consistency Training for Density-Functional-Theory Hamiltonian Prediction. ICML 2024.
- [2] Li, Y. et al. Enhancing the Scalability and Applicability of Kohn-Sham Hamiltonians for Molecular Systems. ICLR 2025.
We sincerely thank the reviewer for the thoughtful, constructive review. Your questions and suggested controls helped us significantly strengthen the empirical analysis, ablations, and theoretical discussion in the paper. We ran controlled experiments from the submitted checkpoint and code to address each of the five weaknesses and three questions. The table below summarizes the new results, and the following sections detail each experiment and the corresponding revisions to the manuscript.