- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
PACT is a structure-based affinity scoring function whose prediction can be decomposed into signed contributions over twelve interaction channels (in pK units), with bounded local corrections from chemistry and pocket context. The main claim is that the explanation is what the model actually computes, not a post-hoc attribution.
Critical strengths
- The prediction adds up exactly from contact-level contributions in affinity units, which is more direct than attention or post-hoc methods.
- The contributions are clearly more stable and reproducible under retraining and small geometric perturbations than most baselines.
- When high-ranked contacts are removed, the change in the score lines up well with the assigned contributions, so the explanation is largely faithful to the model.
Critical weaknesses
The contributions are reported in pK units, but the paper never checks whether they correlate with real or approximate interaction energies. There is also no test of whether these contributions are actually useful for downstream design decisions. As written, the evidence mainly shows faithfulness to the model, not physical meaning or practical value.
Questions for authors
Could the authors report the correlation between several key channel-level contributions and external energy estimates such as MM/PBSA on a subset of complexes?
5 comments
Experiment 1 - uPA [3]. The starting compound carries a chlorine atom at position 6, and the authors replaced it with larger groups that reach into a side pocket. A pyrimidine ring forms a hydrogen bond to Gly219 and keeps it. A second contact to Arg217 depends on how the ligand sits: it is present in one compound and disappears in the next one. We assigned every contribution only to the added group, so the unchanged part of the molecule cannot produce them.At Gly219, a hydrogen bond appears with the pyrimidine ring and stays. At Arg217, it appears with the first methoxy substitution and then drops by 95.8% for the second substitution, as the interaction distance increases from 3.425 to 4.262 A. The molecule grows by two heavy atoms and gains two contacts at Arg217. Size and occupancy therefore both increase, and the named channel decreases. The test is which channel carries the change, and whether the direction is correct.
| Compound / PDB | 6-substituent | Gly219 H-bond, pK | Arg217 H-bond, pK |
|---|---|---|---|
| 5 / 5ZA7 | Cl | 0 | 0 |
| 25 / 5ZAG | pyrimidine | 0.0538 | 0 |
| 26 / 5ZAH | methoxypyrimidine | 0.0545 | 0.0250 |
| 27 / 5ZAJ | dimethoxypyrimidine | 0.0500 | 0.0011 |
Experiment 2 - BACE1 [4]. In this experiment, all molecules in the series should bind BACE1 by forming the same key interactions with aspartic acids. Across the six held-out complexes, the ionic term on Asp93 and Asp289 varies by only 1.4% (stable across molecules), while measured potency resulting from other substitutions spans 1.95 pK. On the other hand, the aromatic contribution calculated by PACT, which was supposed to change, moves from 0.026 to 0.151 pK. The same holds outside the series. In all 56 BACE1 test complexes, the ionic term is non-zero only on a catalytic aspartate. It is not fixed to aspartate, though: arginine and lysine carry 27% of the ionic signal in the test set.
PACT behaves correctly in both experiments, showing that the model can provide valuable insights about binding interactions in real drug design campaigns. We hope that this addresses the Reviewer's concerns, and we are glad to answer any further questions.
[1] Kalinowsky, L. et al. A diverse benchmark based on 3D matched molecular pairs for validating scoring functions. ACS Omega 3, 5704-5714 (2018).
[2] Korlepara, D. B. et al. PLAS-20k: extended dataset of protein-ligand affinities from MD simulations for machine learning applications. Scientific Data 11, 180 (2024).
[3] Buckley, B. J. et al. 6-Substituted hexamethylene amiloride (HMA) derivatives as potent and selective inhibitors of the human urokinase plasminogen activator for use in cancer. J. Med. Chem. 61, 8299-8320 (2018).
[4] Mandal, M. et al. Design and validation of bicyclic iminopyrimidinones as beta amyloid cleaving enzyme-1 (BACE1) inhibitors: conformational constraint to favor a bioactive conformation. J. Med. Chem. 55, 9331-9345 (2012).
Dear Reviewer An95,
Thank you for your focused and constructive review. Your suggestion to compare our contributions with MM/PBSA components became a new section, and your point about practical value led us to test the method on an external benchmark and on two published medicinal chemistry campaigns. All changes are marked in blue in the revised paper.
As the discussion period closes soon, we would greatly value your thoughts on these additions. If anything is unclear or you would like more detail, we would be glad to respond promptly.
Thank you again,
The Authors
We thank the Reviewer for their time and for the questions and suggestions. We ran new experiments and included them in the revision.
Q1. Correlation between channel contributions and external energy estimates.
We thank the Reviewer for suggesting we evaluate our method against MM/PBSA. We note that PACT’s interaction channels are predefined physical forms, and its learned contributions are fitted to experimental binding affinity, not specific interaction energies. We test these predicted interactions against a subset of the PLAS-20k dataset (new Section 5.5), which reports MM/PBSA components computed from molecular dynamics trajectories. There are 888 complexes in our test set that appear there with stable component values. We assigned each of the twelve channels to one component, then we applied this mapping and computed the correlation (Appendix D.8). The correlation values in every row have the expected sign, which does not change after removing the size effect. All values have p<10−3.
MM/PBSA reports only four components, so four groups are all this method provides. While the correlation is not perfect, the direction is consistent across all components, even though PACT predicted its channels from a static PDB frame rather than an MD trajectory.
W1. Practical value for design decisions.
In the original submission, the matched molecular pairs experiment best demonstrates our model's practical utility. We first used an external benchmark, the 3D-MMP set of Kalinowsky et al. [1], which contains 99 matched pairs, 178 structures and 33 target clusters. For each matched pair, we recorded what the edit added or removed. We read it from the same input atom features the channels use without checking any channel’s value. Labels count input atom features and, based on them, assign classes. We then tested whether the responsible channel moves as the chemistry implies. It does in 0.649 [0.576, 0.728] of 171 edits. It is important to add that for some complexes, the protein changes, so it might affect the prediction. Across different heavy-atom counts, the result stays flat (0.618, 0.771 and 0.618). It also achieves 0.652 on the complexes that are only in the test set. This shows that PACT performance is neither from size nor from training subset membership. New Appendix B.4 gives the protocol and per-class results.
In the current revision, to further address the question of practical usefulness of the model, we retrospectively analyzed two published lead optimization papers. Each describes a series of compounds from one study. The authors made one modification at a time and solved a crystal structure for every step. They also state which interaction each change was meant to create or remove. We have performed two experiments. The first is based on a study in which the authors report a series of modifications, one of which introduces an interaction that disappears when an additional functional group is attached. The channel corresponding to this interaction should fire only for the modification that forms the new interaction. The second experiment is based on a study that analyzed a group of BACE1 inhibitor analogs. Potency rises about ninety-fold across the series, but the interactions with the molecular core should remain the same. Both are described in detail in the new appendix D.11.
(continued in the next response)