acceptodds
Under review as a conference paper at ICLR 2027

Hessian Matching for Machine-Learned Coarse-Grained Molecular Dynamics

Abstract

Coarse-grained (CG) molecular dynamics enables simulations of atomic systems such as biomolecules at timescales inaccessible to all-atom (AA) methods, but existing CG neural potentials trained via force matching capture only the gradient of the free-energy surface, leaving its curvature unsupervised. We introduce a framework that augments force matching with stochastic Hessian-vector product (HVP) matching, instilling second-order curvature information into CG potentials without constructing the full Hessian. We derive a decomposition of the target CG Hessian into a model-independent projected AA Hessian, precomputed once before training, and a model-dependent covariance correction computed online at negligible cost. We then construct an unbiased stochastic estimator of the Hessian-matching objective by using random probe vectors. We evaluate our method by training on a benchmark set of nine fast-folding proteins and gauging agreement of the learned free-energy landscapes with reference data along the directions of slowest collective motion using time-lagged independent component analysis. HVP matching improves slow-mode accuracy over force matching for all but one of our benchmark proteins. It also sharpens local structure, reducing bond-length error on eight of nine proteins, and in several cases by an order of magnitude. Our results demonstrate that higher-order physical supervision is a practical path to more accurate CG potentials for biomolecular simulation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
coarse-grained molecular dynamics, machine-learned potentials, force matching, Hessian-vector products, neural network potentials, protein folding, free-energy surface, graph neural networks, molecular simulation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Vx2HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2(−1)
Soundness
2
Read review
Paper summary

This paper presents Hessian-vector product (HVP) matching to train a coarse-grained potential for molecular dynamics. The primary goal is to jointly supervise the gradient and curvature of the free energy surface, which previous force matching methods fail to capture. Through theoretical derivation, the target CG Hessian is decomposed into two terms: the first is precomputed before training, while the second is computed online at negligible cost. The in-distribution evaluations on 9 fast-folding proteins demonstrate that the Hessian correction generally improves performance in both distributional similarity and structural validity over force matching methods, with acceptable computational overhead during training.

Critical strengths
  • Supervising the curvature of the free energy surface is a promising direction, and through theoretical derivation, the authors develop an effective training method with manageable computational overhead, which is critical for the practical utility of the algorithm.
  • The experiments, covering both in-distribution (Section 4) and out-of-distribution settings (Appendix C), are sufficiently thorough to demonstrate the effectiveness of the proposed method.
Critical weaknesses
  • The writing and organization of the paper could be improved to some extent. (i) The "Related Work" section could be presented as a separate section following the Introduction, while the contributions could be moved to the end of the Introduction for better readability. (ii) The "Discussion" section is somewhat overly long, especially given that the main text does not fully use the 9-page limit. (iii) The "Experiments" section is not well organized and is therefore difficult to follow. For example, the "Model variants" subsection includes baselines, hyperparameters, computational infrastructure, and training-cost comparisons. These could be separated into different paragraphs, with some details moved to the Appendix for better coherence. Similarly, the "Evaluation" subsection mixes evaluation metrics with experimental results, which could also be presented separately. In addition, the out-of-distribution evaluation in Appendix C is important and would be better placed in the main text.
Questions for authors
  1. As noted in the Weakness section, my major concern is that the writing and organization of the paper have not yet reached the standard expected for ICLR. If the authors carefully revise the manuscript, I would be happy to reconsider and potentially raise my score during the rebuttal stage.
  2. I have some concerns about the correspondence between Eq. (9) and Eq. (10). The paper states that Eq. (10) is obtained by multiplying both sides of Eq. (9) by vkv_k. However, (i) the first term in Eq. (9) is an ensemble-averaged projected Hessian, whereas the ensemble averaging appears to be omitted in the first term of Eq. (10). (ii) The second term in Eq. (9) depends only on FAAF_{\mathrm{AA}} and is independent of the CG model, while Eq. (10) introduces a δJ\delta J term involving FNNF_{\mathrm{NN}}. Could the authors please clarify these points?
  3. Equation (16) suggests that HVP matching requires at least one additional autograd operation compared with force matching, which is roughly equivalent to an extra backpropagation step. However, Lines 332–333 report only about a 16% increase in training time. Could the authors explain the reason for this relatively small overhead?
  4. In Table 1, for several systems such as Chignolin, Protein B, and Homeodomain, using only the AA Hessian projection appears to yield better performance in the TICA space than using the full loss. Could the authors explain why this occurs?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Vx2H (1/2)

We thank the reviewer for the clear and constructive suggestions, and for the willingness to reconsider the score. We have made the requested changes in the revised PDF, with one exception (the out-of-distribution study), which we explain below.

Writing and organization (Weakness, Q1).

(i) Related Work is now its own section (Section 2), right after the Introduction, and the contributions are listed at the end of the Introduction.

(ii) The Discussion is shorter. It is now a brief Conclusion with a short Limitations paragraph (Section 7).

(iii) The experimental setup (Section 5) is split into clear parts: Data and benchmark; Models and baselines; Training details; Training cost; Evaluation metrics. Results are in a separate section (Section 6).

  • We thank the reviewer for the suggestion to move the out-of-distribution study into the main text. After discussion, we decided to keep it in Appendix C. This experiment is less controlled than the in-distribution one (a different, larger training set), and its results are noisier and more mixed, so we consider it preliminary and do not want the main conclusions to rest on it. We refer to it briefly in the main text, with a one-sentence summary (Section 6) and a pointer to Appendix C.
  • The analysis of the estimator is now in the main text (Section 4.1), with details in the new Appendix D.

Q2. Correspondence between Eq. (9) and Eq. (10).

Thank you for this comment; both observations are correct. (i) Eq. (10) of the submission is a single-frame quantity, not the ensemble average. Exactly as in force matching, the averaging is done by the squared loss over many frames. Fitting noisy targets with a squared loss returns their average. (ii) The covariance in Eq. (9) indeed depends only on all-atom forces. But computing a covariance requires the mean force, which is unknown, so we use the network force FNN\mathbf F_{\mathrm{NN}} (which force matching trains to approximate the mean force FCG\mathbf F_{\mathrm{CG}}) as the estimate. This makes the practical target depend on the model and introduces a bias of exactly βεεT\beta \boldsymbol\varepsilon \boldsymbol\varepsilon^T, where ε\boldsymbol\varepsilon is the model's force error. The bias vanishes when force matching converges. In the revision, the estimator H~CG\tilde{\mathbf H}_{\mathrm{CG}} is defined explicitly (Eq. 12) and its average is stated (Section 4.1); the analysis and a new test on a model with a known exact Hessian are in Appendix D. We explain this step in more detail in our response to Reviewer pzQP (Q3), and give the full bias analysis and the test in our response to Reviewer nFXT (Part 1).

AuthorsOpenReview ↗

Response to Reviewer Vx2H (2/2)

Q3. Why is the training overhead only about 16%?

Upon checking, we realized that we did not rigorously measure and record the training time consistently across the three variants, so the 16% figure is not properly supported. We have therefore removed these timing claims from the revised paper.

Instead, the revision describes the cost by counting operations, which does not depend on timing measurements (Section 5, Training cost). Each force-matching step already needs one backward pass to get the forces from the energy and a second backward pass to get the parameter gradients. HVP matching adds KK further backward passes (one per probe, K=8K = 8 here), and the parameter-gradient pass then also runs through them. So HVP matching costs more per step than force matching, and this extra cost grows with KK. The covariance correction (Term 2) only needs a few vector operations on forces that are already computed. Term 1 needs 2K2K all-atom force evaluations per frame, done once before training and reused. We now state these costs plainly and no longer describe the overhead as small.

Q4. Why does Term 1 alone sometimes beat the full loss in TICA space (chignolin, Protein B, Homeodomain)?

We see three reasons.

  1. Some of these gaps are within the error bars. For Protein B (TIC0: 1.77±1.141.77 \pm 1.14 vs. 1.96±0.391.96 \pm 0.39) and Homeodomain (TIC0: 2.33±0.502.33 \pm 0.50 vs. 2.50±1.162.50 \pm 1.16; TIC1: 2.24±0.902.24 \pm 0.90 vs. 3.29±1.973.29 \pm 1.97), the two variants are not clearly different. Chignolin is a clear case (TIC0: 0.40±0.110.40 \pm 0.11 vs. 1.23±0.241.23 \pm 0.24).
  2. Term 2 is much noisier. It is estimated from a single frame, and the Hessian part of the training loss is roughly 75 times larger with the covariance term than without it (Fig. 2, Appendix B). This noise, plus the bias early in training, can outweigh the benefit of the more correct target in some systems.
  3. Term 1 alone is a too-stiff target. On the model with a known exact answer (Appendix D), Term 1 alone is up to 47% too stiff in the softest mode. A too-stiff target is not correct physics, but it can act as a stabilizing regularizer. That can improve the simulated distribution for some systems even though the learned Hessian is less accurate.

In the revision, we no longer present FM+AAp\mathrm{AA}_p as an approximation of the CG Hessian. We present it as an ablation (Section 5), explain these points (Section 6), and note that each variant was trained with a single training seed, so the error bars reflect MD sampling only (Sections 5 and 7).

Thanks for the detailed rebuttal and the revisions to the manuscript. However, several of my concerns remain insufficiently addressed.

  1. Writing and organization. I have reviewed the revised manuscript and acknowledge the improvements to the experimental setup section. Nevertheless, I believe the overall presentation still requires substantial improvement. For example, (i) the related work could be organized into thematic subsections rather than presented as a largely continuous discussion, (ii) the main text contains only one figure and one table, making several lengthy textual explanations difficult to follow intuitively, and (iii) the conclusion occupies an entire page, whereas the experimental section, which is arguably more important for evaluating the proposed method, spans only approximately two pages. I encourage the authors to learn from the organization and presentation of previously accepted conference papers.
  2. Equations (9) and (10) in the original submission. This concern has been adequately addressed.
  3. Training overhead. The authors acknowledge in their rebuttal that the original training time measurements were not recorded rigorously. It raises broader concerns about the reliability and reproducibility of the experimental evaluation, particularly whether the training procedures and computational budgets were consistently and fairly controlled across methods.
  4. Effectiveness of the proposed method. Regarding Table 1, the authors explain that Term 2 produces excessively noisy supervision, whereas Term 1 provides an overly stiff training target. However, these explanations raise further questions about the effectiveness and necessity of the proposed formulation. The additional training complexity introduced by both terms does not appear to be justified by consistent performance improvements in Table 1. Therefore, I remain unconvinced that the increased optimization difficulty is warranted.

Overall, I will maintain my original score.

Reviewer_nFXTOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a Hessian-matching framework for learning coarse-grained (CG) molecular potentials by augmenting conventional force matching with second-order curvature supervision. The authors derive a CG free-energy Hessian identity consisting of a projected all-atom Hessian and a force-covariance correction. To avoid explicitly computing full Hessian matrices, they introduce stochastic Hessian-vector product (HVP) matching, with precomputed all-atom targets and an online covariance correction. Experiments on nine fast-folding proteins show improvements in TICA-projected distribution agreement and local structural statistics compared with force matching alone, with modest reported training overhead.

Critical strengths

The paper aims to address a meaningful limitation of conventional force matching by incorporating curvature information into CG potential learning. The decomposition of the CG Hessian into projected atomistic curvature and force covariance provides a theoretically motivated framework that could benefit future CG modeling.

Critical weaknesses
  1. The proposed Hessian supervision does not provide an unbiased estimate of the true CG Hessian. The theoretical CG Hessian involves a conditional force covariance, but the implementation replaces it with a single-snapshot outer product of the force residual relative to the learned force. Its expectation contains an additional model-dependent bias proportional to the squared mean-force prediction error. Consequently, the training target generally differs from the true CG Hessian, particularly before force matching has converged. The claimed unbiasedness of random HVP probes does not resolve this discrepancy. This fundamentally undermines the theoretical justification of the proposed method, since its central contribution is physically grounded second-order supervision. The authors should establish the validity of the practical estimator, ideally through analytical systems with known CG free-energy Hessians.

  2. The experimental evaluation does not substantiate the claimed improvements in molecular dynamics. The main evidence relies on KL divergences of TICA-projected distributions and structural statistics, which primarily measure equilibrium conformational agreement rather than dynamical fidelity. Matching distributions along slow collective coordinates does not establish accurate transition rates, relaxation timescales, or folding/unfolding kinetics. Without evaluating these properties, it remains unclear whether Hessian matching improves the actual molecular dynamics or merely certain equilibrium statistics. This is particularly consequential given that improving the fidelity of CG molecular simulations is a central motivation of the paper. Quantitative comparisons of implied timescales, transition probabilities, or mean first-passage times are necessary to support the broader claims.

Questions for authors
  1. Validity of the Hessian estimator. In Eq. (10), the conditional force covariance from Eq. (9) is replaced by the outer product of a single-frame force residual relative to the model prediction. Since the expectation of this quantity includes an additional term proportional to the squared mean-force prediction error, under what assumptions is the resulting Hessian target unbiased? Can the authors theoretically or empirically quantify this bias and demonstrate that it does not undermine the proposed curvature supervision?

  2. Validation of dynamical fidelity. The reported TICA-based KL divergences characterize projected conformational distributions but do not directly establish accurate molecular kinetics. Have the authors evaluated transition rates, implied timescales, or mean first-passage times against the all-atom reference simulations? Such results would materially affect whether the reported improvements support the paper's broader claims about more accurate CG molecular dynamics.

2 comments
AuthorsOpenReview ↗

Response to Reviewer nFXT (1/2)

We thank the reviewer for a careful and critical review. Both points are fair, and the first led us to a clearer analysis of the method.

Part 1: Is the Hessian target unbiased? (W1, Q1)

The reviewer is correct. The random probes are unbiased for a given target, but the target itself is biased whenever the model's force is wrong. We have corrected this claim in the paper and added the analysis below (Section 4.1 and Appendix D).

How large is the bias?

Our per-frame Hessian target (Eq. 10 of the submission) is now written explicitly as the single-frame estimator H~CG=ΞFHAAΞFT−β δJ δJT\tilde{\mathbf H}_{\mathrm{CG}} = \boldsymbol\Xi_F \mathbf H_{\mathrm{AA}} \boldsymbol\Xi_F^T - \beta \thinspace \delta\mathbf J \thinspace \delta\mathbf J^T (Eq. 12 of the revision), i.e., Term 1 minus Term 2. Term 1 needs no correction, as each frame gives a noisy but unbiased sample of the first term of Eq. 9, just as force matching uses noisy forces. The bias comes from Term 2. Let f≡ΞFFAA\mathbf f \equiv \boldsymbol\Xi_F \mathbf F_{\mathrm{AA}} be the projected all-atom force in a frame, FCG(R)=⟨f⟩R\mathbf F_{\mathrm{CG}}(\mathbf R) = \langle \mathbf f \rangle_R its average at fixed CG configuration R\mathbf R, ΣR=⟨(f−FCG)(f−FCG)T⟩R\boldsymbol\Sigma_R = \langle (\mathbf f - \mathbf F_{\mathrm{CG}})(\mathbf f - \mathbf F_{\mathrm{CG}})^T \rangle_R the true conditioned force covariance in Eq. 9, δJ=f−FNN\delta\mathbf J = \mathbf f - \mathbf F_{\mathrm{NN}} the force residual, and ε(R)=FNN(R)−FCG(R)\boldsymbol\varepsilon(\mathbf R) = \mathbf F_{\mathrm{NN}}(\mathbf R) - \mathbf F_{\mathrm{CG}}(\mathbf R) the model's error in the mean force. Expanding δJ δJT\delta\mathbf J \thinspace \delta\mathbf J^T and using the fact that force fluctuations average to zero at fixed R\mathbf R (the same step that justifies force matching) gives

⟨δJ δJT⟩R=ΣR+εεT,\langle \delta\mathbf J \thinspace \delta\mathbf J^T \rangle_R = \boldsymbol\Sigma_R + \boldsymbol\varepsilon \boldsymbol\varepsilon^T,

so on average the estimator equals HCG−βεεT\mathbf H_{\mathrm{CG}} - \beta \boldsymbol\varepsilon \boldsymbol\varepsilon^T instead of the true CG Hessian HCG\mathbf H_{\mathrm{CG}}. The bias is therefore set entirely by the force error, so it shrinks as force matching improves. Now suppose the network has learned the exact CG potential everywhere, not just the correct forces at the sampled configurations (assuming it is flexible enough to represent it). Then its force equals the mean force, so the force error ε\boldsymbol\varepsilon is zero and the bias disappears. The average Hessian target is then the true CG Hessian, which is also the model's own Hessian. So, on average, neither the force loss nor the Hessian loss pushes the model away from the exact answer. Note that matching the mean force only at the sampled configurations is not enough. With finite data, a model can do that and still have the wrong curvature between samples, which is the gap Hessian matching aims to fill. Because the network is a nonconvex function of its parameters, we cannot guarantee that training reaches this point. This differs from force matching with a linear basis set, where the forces depend linearly on the parameters and, with a complete basis, the variational principle guarantees convergence to the exact solution (Noid et al., J. Chem. Phys., 2008). In addition, because the target depends on the model, training could be drawn toward a wrong answer instead.

A variational estimate of the covariance.

The same quantity gives a variational principle for the covariance, like the one behind force matching. For any model force,

⟨δJ δJT⟩R=⟨(f−FNN)(f−FNN)T⟩R=ΣR+εεT,\langle \delta\mathbf J \thinspace \delta\mathbf J^T \rangle_R = \langle (\mathbf f - \mathbf F_{\mathrm{NN}})(\mathbf f - \mathbf F_{\mathrm{NN}})^T \rangle_R = \boldsymbol\Sigma_R + \boldsymbol\varepsilon \boldsymbol\varepsilon^T,

which is never smaller than the true covariance and equals it only when the model force is the mean force. If we add up the diagonal entries of this estimate and average over configurations, we get exactly the force-matching loss (times 3N3N). So lowering the force-matching loss shrinks the extra part, εεT\boldsymbol\varepsilon \boldsymbol\varepsilon^T, and brings the estimate closer to the true covariance. Force-matching training already does this. This suggests an additional simple variant: first train with force matching alone, then compute the Term 2 targets once from that model's forces on the training configurations, and keep them fixed while training continues with both losses. The target then no longer changes during training (its remaining bias is set by that model's force error), and Hessian matching becomes ordinary regression. We present this as a recommendation, and we did not run it.

AuthorsOpenReview ↗

Response to Reviewer nFXT (2/2)

Test with a known exact answer.

As the reviewer suggested, we tested the method on a Gaussian network model (GNM) (Bahar et al., Fold. Des., 1997; Haliloglu et al., Phys. Rev. Lett., 1997), for which the exact CG potential and Hessian are known in closed form (Foley, Shell & Noid, J. Chem. Phys., 2015; Kidder & Noid, J. Chem. Phys., 2024). We took the 93 heavy atoms of a chignolin structure from our dataset and connected every pair closer than 5 Å by an identical spring, with the spring strength chosen so that Term 1 has the same typical size as in our real chignolin data. Keeping only the 10 Cα\mathrm C_\alpha atoms (30 coordinates), the exact CG potential is still quadratic, W=12RTHCGRW = \tfrac12 \mathbf R^T \mathbf H_{\mathrm{CG}} \mathbf R, where R\mathbf R is the Cα\mathrm C_\alpha displacement. We drew 20,000 independent equilibrium configurations directly from the model (no MD) and trained a CG model of the same form, W=12RTARW = \tfrac12 \mathbf R^T \mathbf A \mathbf R with a learnable symmetric matrix A\mathbf A, using the same losses and 8 random probes per frame as in the paper.

  • Eq. 9 reproduces the exact CG Hessian to within 0.36%.
  • The size of the measured bias agrees with β∣ε∣2\beta \vert \boldsymbol\varepsilon \vert^2 to within about 1% (above the sampling-noise level).
  • At wHVP=0.01w_{\mathrm{HVP}} = 0.01 (our setting), the bias falls from 5.1∥HCG∥5.1 \Vert \mathbf H_{\mathrm{CG}} \Vert at the start of training to 3×10−4∥HCG∥3 \times 10^{-4} \Vert \mathbf H_{\mathrm{CG}} \Vert at the end, most of it within the first few epochs, after which the Hessian term reduces the Hessian error below that of force matching. The final Hessian error is 0.51% for the full objective, 0.73% for force matching alone, and 4.70% for Term 1 alone, which converges to a too-stiff Hessian.
  • With a larger weight, training can go wrong, with the Hessian error growing instead of shrinking: at wHVP=0.1w_{\mathrm{HVP}} = 0.1 from a cold start (but not when started from the force-matching solution), and at wHVP=1w_{\mathrm{HVP}} = 1 from every start we tried.

This test checks that the target and its bias behave as derived, and it is not meant to measure the benefit on proteins. For a harmonic model, force matching alone already learns the Hessian well.

Changes to the paper. We now (i) define the estimator H~CG\tilde{\mathbf H}_{\mathrm{CG}} explicitly and state that it is correct on average only when the model's force equals the mean force (Section 4.1); (ii) recommend wFM≫wHVPw_{\mathrm{FM}} \gg w_{\mathrm{HVP}} and starting from force matching (Section 4.1); (iii) give the bias analysis, the variational covariance estimate, and the GNM test in the new Appendix D; and (iv) no longer describe the estimator as unbiased in the abstract or contributions.

Part 2: Dynamics vs. equilibrium (W2, Q2)

We agree that KL divergences along TICA coordinates measure equilibrium distributions, not kinetics. Like force matching and the vast majority of CG models, including ML CG models (CGnet, CGSchNet, Majewski et al. 2023, Charron et al. 2025), our method targets the equilibrium CG free-energy surface. Only a few ML CG models address dynamical properties (e.g., Fu et al., TMLR, 2023). Instead, we only use CG MD to sample the equilibrium probability distribution, rather than to model real dynamics. Coarse-graining removes friction and smooths the landscape, so CG dynamics typically run faster than all-atom dynamics unless a friction or memory model is added, which is outside our scope. We use TICA only to choose the coordinates on which we compare equilibrium distributions, namely the slowest ones, which are commonly viewed as the most important (such as folding and unfolding) and are also the hardest to sample.

Changes: we replaced “slow collective dynamics” with “free-energy surface” and “equilibrium distributions along the slowest coordinates”, removed the claim of more faithful slow dynamics, noted in the Introduction that a more accurate CG potential does not necessarily improve dynamical quantities, and added kinetics to the Limitations (Section 7).

Reviewer_pzQPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces a second order correction to force-matching (FM) used for defining collective variables (coarse-graining (CG)) in molecular dynamics. They argue that the real goal of FM is to learn the free energy landscape. Vanilla FM only matches the gradients of the free energy and misses the curvature info. They add hessian corrections to FM and see significant improvemnts in CG variables reproducing behavior of all-atoms (AA) simulations. They use a GNN-based (SchNet) parametrization of the CG variables, using the AMBER force-field and benchmark their method on 9 small and fast-folding proteins (Aghili 2025 benchmark).

Critical strengths
  1. Great idea to supplement force-matching with Hessian info.
  2. The precomputable first term of the Hessian correction makes this much more attractive.
  3. The Hessian-vector product is a great way to make it tractable.
  4. Improvements on the small proteins seem impressive.
Critical weaknesses

None

Questions for authors
  1. Why only random vectors? Why not PCA on Hessian or something? In high dimensional space, would random vectors not miss some important direction? Ex: slow-modes generally have low spatial frequency, but random vectors have high spatial freqs. So random projections may not cover the slow subspace that well.
  2. Have you tried learning the vkv_k instead of random?
  3. Eq 10 doesn't directly follow from eq 9. Where does the -FNNF_{NN} in eq 11 come from? The original force-matching loss, eq 12? Something is confused here. δJ\delta J makes sense from eq 12, but not as part of HCGvkH_{CG} v_k.
  4. lines 254-260: WNNW_{NN} not properly defined. I assume this is the NN parametrized energy function (force field) to be learned? The GNN arch used for modeling WNNW_{NN} or FNNF_{NN} is not described before in the text.
  5. lines 360-365: Interesting. I was also curious if the Cov term could have negative effects for slow-modes because a large part of it will be side-chain correlations, invisible in the CαC_\alpha-bead CG basis.
  6. Was hessian correction also proposed in classical force-matching? Independent of the NN force-field, the Hessian correction idea seems very relevant to the physics problem.
2 comments
AuthorsOpenReview ↗

Response to Reviewer pzQP (1/2)

We thank the reviewer for the positive assessment and the thoughtful questions.

Q1. Why random probes? Could random vectors miss slow directions? We use random probes for two reasons. First, they let a few probes stand in for the whole Hessian. On average, the error measured along random directions is proportional to the total Hessian error. This holds because the probes have no preferred direction (we normalize Gaussian vectors with independent components). If we favored some directions, the loss would no longer measure the total Hessian error. Second, PCA of the Hessian is not available to us. Building or diagonalizing the CG Hessian is the cost we aimed to avoid, and the target Hessian changes from frame to frame.

In terms of coverage, a Gaussian random vector is not necessarily biased toward high spatial frequencies. On average, it puts equal weight on every normal mode, slow or fast. However, the reviewer's concern may arise in a slightly different form. Typically, the slow modes span a low-dimensional subspace (dimension mm), while the fast modes make up most of the full space of d=3Nd = 3N CG coordinates. Because most modes are fast, most of a probe's weight lands on fast modes, and the slow subspace receives only a fraction m/dm/d of it. Each frame uses its own probes, so all directions are covered across the training set, but the slow directions receive a small share of the signal. One could mix random probes with probes aimed at slow directions, such as the TICA directions from the reference data. The loss would then no longer measure the plain Hessian error, but rather would weight errors in the slow directions more heavily. This may be useful, but it is a different objective. We now mention this in the Limitations paragraph (Section 7); we have not tested it in the paper.

Q2. Have you tried learning the probes vk\mathbf v_k? We have not tried this, but it is an interesting direction. However, there is a practical obstacle. The expensive part of the target (Term 1) is computed once, before training, with all-atom force evaluations for fixed probes. If the probes changed during training, Term 1 would have to be recomputed with the all-atom force field at every step, which would remove the main cost advantage of the method. Learned probes would also change which objective is being minimized. We mention this as future work (Section 7).

Q3. Eq. (10) does not follow directly from Eq. (9). Where does −FNN-\mathbf F_{\mathrm{NN}} in Eq. (11) come from? Thank you for this question, as this step was not explained well. Eq. (9) is an exact identity, but both of its terms are averages over all-atom configurations that map to the same CG configuration R\mathbf R. During training we only have single frames, so, just as in force matching, we replace each average with a single-frame sample and let the squared loss do the averaging. This works because fitting noisy targets with a squared loss returns their average. For Term 1, each frame gives one sample of the projected all-atom Hessian. Term 2 is a covariance, representing the spread of the projected force around its mean. The true mean force FCG\mathbf F_{\mathrm{CG}} (Eq. 7) is not known, but force matching trains the network force FNN\mathbf F_{\mathrm{NN}}, the force from the neural-network potential at the given CG configuration, to approximate FCG\mathbf F_{\mathrm{CG}}, so we use it as our estimate. Thus, δJ=ΞFFAA−FNN\delta\mathbf J = \boldsymbol\Xi_F \mathbf F_{\mathrm{AA}} - \mathbf F_{\mathrm{NN}} is the deviation of this frame's projected force from the current estimate of the mean force.

In the revision (Section 4.1), this single-frame estimator is written out explicitly as H~CG=ΞFHAAΞFT−β δJ δJT\tilde{\mathbf H}_{\mathrm{CG}} = \boldsymbol\Xi_F \mathbf H_{\mathrm{AA}} \boldsymbol\Xi_F^T - \beta \thinspace \delta\mathbf J \thinspace \delta\mathbf J^T (Eq. 12), and we state exactly what its average is:

⟨H~CG⟩R=HCG(R)−β εεT,ε=FNN−FCG.\langle \tilde{\mathbf H}_{\mathrm{CG}} \rangle_R = \mathbf H_{\mathrm{CG}}(\mathbf R) - \beta \thinspace \boldsymbol\varepsilon \boldsymbol\varepsilon^T, \qquad \boldsymbol\varepsilon = \mathbf F_{\mathrm{NN}} - \mathbf F_{\mathrm{CG}}.

So the target is correct on average apart from one term set by the model's force error, and that term disappears when force matching has converged. Details are in the new Appendix D and in our response to Reviewer nFXT (Part 1).

AuthorsOpenReview ↗

Response to Reviewer pzQP (2/2)

Q4. WNNW_{\mathrm{NN}} is not defined and the GNN is not described by this line.

Correct: WNNW_{\mathrm{NN}} is the CG potential energy given by the neural network, and FNN=−∇RWNN\mathbf F_{\mathrm{NN}} = -\nabla_R W_{\mathrm{NN}} is its force. We now define these, together with the Hessian HNN\mathbf H_{\mathrm{NN}}, at the start of Section 4 (Methodology) and describe the network (the SchNet-based GNN from TorchMD-Net) there, before it is used; full architecture details are in Section 5.

Q5. Could the covariance term hurt the slow modes, since much of it comes from side-chain motion that is invisible in the Cα\mathrm C_\alpha representation?

Physically, the covariance term measures how much the atomic forces on the CG sites fluctuate while the CG configuration is held fixed. These fluctuations come from the coupling between the CG coordinates and the remaining degrees of freedom. So side-chain motion contributes only to the extent that it is coupled to the Cα\mathrm C_\alpha positions. If the all-atom energy had no terms coupling the CG coordinates to the other degrees of freedom, the covariance term would vanish. The all-atom Hessian alone (Term 1) describes the curvature as if the other degrees of freedom were held fixed. Allowing them to move makes the CG landscape softer, and the covariance term represents that softening, preventing the target from being too stiff. We checked this on a system where the exact answer is known (a Gaussian network model (GNM) of chignolin; Appendix D and our response to Reviewer nFXT). There, Term 1 alone overestimates the curvature of the softest internal mode by 47% and of the stiffest mode by only 4%. So the error from dropping the covariance term is largest in the soft, slow directions. How strongly the Cα\mathrm C_\alpha atoms are coupled to the rest of the protein may also differ between proteins, which may partly explain why the two HVP variants rank differently across our benchmark.

That said, the reviewer is right that the term can hurt in practice, for a different reason. We estimate it from single frames, so it is noisy, and it is biased while the force error is still large. This may partly explain why Term 1 alone sometimes does better (see also our response to Reviewer Vx2H, Q4).

Q6. Was a Hessian correction proposed in classical force matching?

To our knowledge, the free-energy Hessian has not been used as a training target in force matching. The identity itself (Eq. 9) is closely related to known results. For Gaussian (harmonic) models, the same two-term structure appears naturally when some degrees of freedom are integrated out. The effective model for the kept coordinates follows from splitting the covariance matrix into kept and removed parts (the Schur complement of a matrix inverse), and the same structure also arises in least-squares problems. Expressions of this form appear in Diggins et al. (J. Chem. Theory Comput., 2019) and in the analysis of Kidder & Noid (J. Chem. Phys., 2024), whose Eq. 54 has a first term corresponding to the averaged projected Hessian and a second corresponding to the force-covariance correction. Section IIIA of that work also expresses the potential of mean force locally in terms of fluctuations at a fixed CG configuration. Exact CG potentials of Gaussian network models have also been analyzed by Foley, Shell & Noid (J. Chem. Phys., 2015), Foley et al. (PNAS, 2020) and Kidder, Shell & Noid (J. Chem. Phys., 2024), and fluctuation information has been used to parameterize elastic-network CG models from atomistic simulations (fluctuation matching; Chu & Voth, Biophys. J., 2006; Lyman, Pfaendtner & Voth, Biophys. J., 2008). We now discuss these connections in Related Work (Section 2).

Relative to this work, our contributions are: (i) an exact expression for the CG Hessian that holds for arbitrary (anharmonic) all-atom potentials, including realistic molecular-mechanics force fields, not only harmonic or normal-mode models, and that extends to nonlinear CG maps (Appendix A); (ii) a practical HVP-based algorithm for training neural CG potentials to match both the gradient and the Hessian of the exact CG potential, together with an analysis of the resulting estimator, including a variational estimate of the force covariance (Section 4.1 and Appendix D); and (iii) validation on a GNM with a known exact Hessian and on nine fast-folding proteins.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.