acceptodds
Withdrawn from ICLR 2027

AffinitySym: Sequence-based Interpretable Protein–Protein Binding Affinity Prediction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Protein–protein binding affinity, often expressed as a change in Gibbs free energy (ΔG), quantifies the stability of a complex. Most predictors require a three-dimensional structure and return a scalar without explaining the estimate. AffinitySym is a multimodal, sequence-first deep neural model with frozen ESM-3 and MINT encoders. Bidirectional cross-attention combines partner sequences with additional equivariant graph neural network (EGNN) and invariant point attention (IPA) features when a structure is supplied; learned-query attention conditions an autoregressive Transformer decoder that generates 16 executable symbolic formulas. Dual-path training lets the same checkpoint predict from partner sequences alone or from those sequences paired with a supplied structure, making AffinitySym the only compared method to support both inference modes. Formula-conditioned residue attributions identify amino acid residues that contribute to each prediction. We train on the protein–protein subset of PDBbind v2020 and evaluate on two test sets adopted from prior studies: Test Set 1 (82 complexes) and Test Set 2 (166 wild-type and mutant complexes from SKEMPI v2.0). On Test Set 1, the best-observed sequence-only formula achieves an MAE of 1.2982 kcal/mol and a PCC of 0.7034, compared with an MAE of 1.36 kcal/mol and a PCC of 0.629 for the strongest listed baseline. The same checkpoint also performs competitively on Test Set 2, while a separate checkpoint achieves an MAE of 0.9960 kcal/mol and a PCC of 0.8493. Detailed comparisons on both test sets and multi-seed results are presented in the main paper. Frozen encoders reduce training and gradient-memory costs, while local inference lets researchers keep sensitive protein data on their own systems. Residue-level attributions can help prioritize binding-critical residues for drug discovery. Code Availability (Anonymous): https://anonymous.4open.science/r/AffinitySymProject-D964

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein–protein interactions, Protein binding affinity prediction, Protein language models, Explainable AI, Symbolic regression, Graph neural networks, Multimodal learning, Computational biology, AI for science

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fW3iOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose a protein–protein binding-affinity predictor using frozen ESM-3 and MINT together as sequence encoders, with an optional structure branch using EGNN and invariant point attention. An interesting direction is instead of using a conventional scalar regression head, the model generates 16 symbolic experts whose arithmetic expressions operate on learned latent concept vectors. Predictions can be combined using a validation-selected consensus. The authors also use gradient-times-input to attribute an expert prediction to individual sequence or structure residues.

The model is trained on PDBbind complexes. Evaluation uses an DBbind-derived test set and a SKEMPI-derived set containing 26 wild-type and 140 mutant records from 26 base complexes. On Test Set 1, a best-observed sequence-only formula obtains MAE 1.2982 kcal/mol and PCC 0.7034, while the validation-selected top-7 consensus from the same checkpoint obtains 1.2990/0.6806. A separate best-observed formula obtains 0.9960/0.8493 on Test Set 2, although its validation-selected top-12 consensus has MAE 1.3910.

Critical strengths

Strengths of the paper are:

  1. Supporting sequence-only and sequence-plus-structure inference with one checkpoint could be practically useful, especially when reliable complex structures are unavailable.

  2. The paper documents PDBbind filtering, benchmark exclusion, chain assignment, and the four runtime exclusions. These are helpful for reproducibility.

  3. Sections 2.4 and A.7 distinguish checkpoint scoring from final consensus construction and explain that expert ranking and KK are fixed before test inference.

  4. Table 2 exposes a large robustness gap that should be resolved, even though its aggregation is not described clearly enough for direct comparison with Table 1.

  5. The authors state that sequence-only inference cannot distinguish alternative binding arrangements, latent concepts are not predefined physical quantities, and residue scores are model-derived hypotheses rather than causal binding energies. The Test Set 2 peak is also explicitly distinguished from expected validation-selected performance.

  6. The features such as exposing expert predictions, expressions, auxiliary distributions, and residue sensitivities could be useful for exploratory analysis given their reliability is validated.

Critical weaknesses

Following are the weaknesses

  1. On Test Set 1, the highlighted individual formula obtains MAE/PCC 1.2982/0.7034, while the validation-selected top-7 consensus from the same checkpoint obtains 1.2990/0.6806 and still exceeds the strongest listed baseline. On Test Set 2, a different checkpoint formula achieves 0.9960/0.8493, but its validation-selected top-12 consensus has MAE 1.3910. Its not clear how the Table 1 checkpoint, configuration, seed, or individual formula was chosen from the full search. If test performance influenced any of these choices, selecting among 16 experts, multiple checkpoints, seeds, configurations, and modalities would create a large multiple-comparisons advantage.

  2. The development/test separation removes identical PDB codes but does not cluster by sequence identity, protein family, interface similarity, or structural homology. Closely related complexes—or even the same protein paired in a related complex—may therefore occur in training and test. The paper should report nearest train–test sequence identity separately for both partners, family and interface overlap, and performance stratified by similarity.

  3. Test Set 2 is SKEMPI which contains 166 observations but only 26 base complexes. The 140 mutant records are clustered and statistically non-independent within their wild-type parent complexes. Pooled PCC or SPCC across all 166 absolute affinities can be high because of between-complex differences, even if a model assigns nearly the same prediction to every mutant of a given parent. Consequently, the statement that SPCC above 0.83 shows that candidate variants are well ordered is not justified by the reported metric. It is also unclear how the sequence-plus-structure path represents mutants. Are mutated residue identities combined with wild-type coordinates, are mutant structures modeled, or are the same wild-type graph features reused? This must be specified, because the interpretation of “structure-conditioned” mutation prediction depends on it.

  4. The formulas expose a sparse portion of the model’s computation, but don't provide a biochemical or mechanistic explanation. They operate on 32-dimensional learned concepts with expert-specific element-wise coefficients. Their vector-valued output is then reduced by an input-dependent softmax πe(z)\pi_e(z), itself produced by a neural function of the representation. For the single 1WR1 example in Appendix B, the 16 displayed experts reduce to three repeated syntactic families, with similar scales and biases near −10-10: L1–L2 share one template, L3–L8 another, and L9–L16 another. One example is not enough to establish globally low diversity, but it shows that symbolic diversity across samples and runs has not yet been demonstrated. Appendices A.5–A.7 indicate that formula syntax is conditioned on each input’s concept memory and generated at inference, so these are input-specific computational traces rather than a single learned global law of protein binding.

  5. Across Table 2, supplying structure produces essentially no improvement and sometimes worsens both MAE and PCC. The consistency term explicitly trains structure-path predictions toward detached sequence-only predictions, which may discourage the structure branch from changing predictions in response to geometry. The stated motivation is instead that structure enables conformation- and interface-specific prediction, but no experiment demonstrates this behavior.

  6. For every training sample, the regression objective directly supervises up to four experts selected using their error against that sample’s true target. At inference, however, experts are ranked globally on validation data and combined using a fixed top-KK consensus; there is no learned sample-specific router corresponding to the target-based training selection. The paper needs an ablation against uniform all-expert supervision, analysis of which samples each expert specializes on, and a comparison between the globally selected consensus and any label-free learned routing mechanism.

  7. No confidence interval, bootstrap analysis, or paired significance test is reported. It is therefore unclear whether the difference is distinguishable from sampling variation. The quoted baselines also come from separate papers rather than a controlled retraining with the same data, split, labels, and preprocessing which make comparison harder.

Questions for authors
  1. Does the “best-observed” Test Set 1 checkpoint mean best according to validation MAE or best after inspecting Test Set 1? How was its displayed individual formula selected?

  2. What is the expected performance of the exact inference procedure a user would run on a new unlabeled protein pair? One consistent estimator should be identified and used throughout the paper.

  3. Because a new train/validation split is generated for every seed, Table 2 combines initialization variance with split variance. Results on a fixed split across seeds, as well as variation across grouped splits, would be more informative.

  4. How are mutant graph node features constructed for Test Set 2? In particular, are mutant residue identities inserted into graphs with wild-type coordinates and structural descriptors?

Reviewer_MLfUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The machine learning content in this paper appears well motivated and solid. The authors use a mixture of experts approach to predict strength of binding for a ligand bound to a protein, and do this is in such a way that each expert can trace its predictions back to particular residues in sequence. They manage to achieve some gains from previous methods on their benchmarks as compared to previous methods which is terrific, since predicting binding strength is a notoriously difficult problem. My main issues are all with how the benchmark datasets were constructed. In particular, the authors give no overall statistics of the similarity of different proteins or ligands across the training set as compared to the test set, instead treating PDB structures with different 4 letter codes as entirely different. In fact, there are many PDB structures with entirely different 4 letter PDB codes that have identical structure around the binding pocket, or even nearly identical structures globally. Without using some sort of structure alignment program, it is impossible to judge whether or not these results will generalize to binding pockets unseen in training.

Critical strengths

The paper is easy to read

The paper on the benchmarks the authors chose to consider makes some progress on a notoriously difficult problem

The machine learning methods are state of the art, involving mixture of experts, current SOTA protein language models (ESM3 and MINT), and a nice first stab at making predictions interpretable, with a way to map predictions back to particular amino acid residues.

Critical weaknesses

As already mentioned above, a critical weakness of this paper is the benchmarking, where the authors give no overall statistics of the similarity of different proteins or ligands across the training set as compared to the test set, instead treating PDB structures with different 4 letter codes as entirely different. In fact, there are many PDB structures with entirely different 4 letter PDB codes that have identical structure around the binding pocket, or even nearly identical structures globally. Without using some sort of structure alignment program, it is impossible to judge whether or not these results will generalize to binding pockets unseen in training. Being able to predict binding strength even when single amino acids change on either the ligand or the target is already an interesting and challenging unsolved problem, so it is hard to believe that we are looking at a method that can generalize to structures unseen in training. But without some measure of whether a similar binding pocket was seen in training, it is impossible to interpret what this method is able to accomplished.

Questions for authors

What version of PDBBind are you using in your paper? It seems to be an old version, but then it appears nowhere in the references. Are you aware of the recent paper: Li, J., Guan, X., Zhang, O., Sun, K., Wang, Y., Bagni, D., & Head-Gordon, T. (2026). Leak Proof PDBBind: a reorganized data set of protein–ligand complexes for more generalizable binding affinity prediction. The Journal of Physical Chemistry B, 130(2), 730-740.?

Suggestion for additional work: characterize in more depth the sequence and structure similarity of the complexes you consider across training and testing sets. Possibly look at the above Leak Proof PDBBind paper for some ideas of how to measure similarity beyond PDB codes.

Reviewer_QxE4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Most protein-protein binding affinity predictors require a three-dimensional structure and return a scalar without explaining the estimate. To predict ΔG with explanations, the paper proposes AffinitySym, a sequence-first model with frozen ESM-3 and MINT encoders. An autoregressive decoder generates 16 executable symbolic formulas, and formula-conditioned residue attributions identify residues that contribute to each prediction. Dual-path training lets the same checkpoint predict from sequences alone or with a supplied structure. On Test Set 1, the best-observed sequence-only formula achieves an MAE of 1.2982 kcal/mol and a PCC of 0.7034.

Critical strengths
  1. Interpretable binding affinity prediction is an important problem. Most existing methods only give a scalar output. Giving both symbolic formulas and residue-level attributions is a meaningful attempt in this direction. The model also works with sequence only, which is useful when no reliable complex structure is available.
  2. The paper reports multi-seed results and gives detailed data processing steps, which helps readers understand the real performance.
Critical weaknesses
  1. The interpretability is limited. Interpretability is the main motivation of this paper, but the formulas are built on learned latent vectors with no physical meaning. Most of the 16 formulas in Appendix B also look almost the same. Besides, the residue attributions are not checked against any experimental data, such as mutation or hotspot data. So it is hard to see what new biological insight the formulas give.
  2. The main results are not reliable. The headline numbers come from the best single formula of a "best-observed" checkpoint, and it is not clear how this checkpoint was chosen. The multi-seed results in Table 2 are much worse (Test Set 1 MAE about 1.64 and PCC about 0.49), which is worse than Multi_PPBA and ProAffinity-GNN. So the state-of-the-art claim mentioned in the contribution part of Introduction is not supported.
  3. The ablation study shows that the structure branch brings little gain. In Table 2, ESM-3 + MINT without the GNN gets similar results on Test Set 1 and better results on Test Set 2 than the full model. This weakens the claim that structure input gives conformation-aware prediction.
Questions for authors
  1. Do the residue attributions agree with known hotspots or SKEMPI mutation data? Can the latent concepts be linked to any physical properties?
  2. Can you compare with the baselines using the multi-seed mean and the validation-selected consensus, instead of the best single formula?
  3. Can you explain the cases where the structure path gives a better or different prediction than the sequence-only path, and the cases vise versa?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.