- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces InteractBind, a large-scale protein–ligand dataset (~100k pairs derived from the PDB) and benchmark focused on residue-level binding-site localization. It supplies sequence-level interaction maps spanning six major non-covalent interaction types and uses these annotations to test whether model-derived interaction maps recover the residues involved in binding. The benchmark comprises an in-distribution (ID) split together with protein-similarity-controlled out-of-distribution (OOD) splits. Eight existing sequence-based and interaction-aware models are evaluated on both binary binding prediction and localization. The central empirical finding is that models can achieve strong binary performance while still exhibiting limited localization ability.
Critical strengths
- InteractBind supplies a form of residue-level, interaction-type-specific supervision that is largely absent from existing binary-binding or affinity-focused protein–ligand benchmarks, thereby enabling finer-grained evaluation of molecular recognition.
- The task formulation is clear and well-motivated: binary prediction and residue-level localization are assessed separately, with localization metrics that cover both top-ranked residues and overall ranking quality.
- The protein-similarity-controlled OOD splits provide a more realistic stress test of generalization to less familiar targets than random-split accuracy alone; the main result—that binary success does not imply reliable localization—is simple, reproducible, and relevant to the community.
Critical weaknesses
- All reported baselines are existing models trained primarily for binary binding prediction (or related pair-level objectives). They are not trained with a localization objective on the residue-level annotations. Consequently, the low localization scores may largely reflect the absence of direct supervision rather than an inherent inability of the architectures to learn binding sites. Because the benchmark is intended to guide future model design, the lack of at least one supervised localization baseline limits the strength of the claims.
- The ID split treats curated PDB pairs as positives and generates negatives by random re-pairing. It is not stated whether the folds are separated at the protein, ligand, or complex level. The same protein (or a close homolog) can therefore appear with different ligands in both training and test data, potentially inflating both binary and localization numbers. Explicit entity-level leakage statistics and/or stricter entity-separated splits would make the standard results easier to interpret.
- Negative sampling and interaction-label construction may introduce shortcuts or noise. Random re-pairing can yield relatively easy negatives, while the rule-based derivation of non-covalent contacts from crystal structures is sensitive to tool settings, distance cutoffs, and geometric criteria. Harder negatives, inter-annotator or tool-agreement checks, and cutoff-sensitivity analyses would strengthen the reliability of the labels and the trustworthiness of cross-model comparisons.
Questions for authors
- Are the standard (ID) evaluation folds separated by protein, ligand, or complex, or only by pair? What are the rates of protein- and ligand-level overlap between train and test?
- Can results for a model trained directly on the residue-level (and/or interaction-type-specific) annotations be added as a supervised localization baseline?
- How do binary and localization metrics change under hard-negative sampling and under alternative interaction-annotation settings (different tools or cutoffs)?
3 comments
Response to Reviewer qoYe [2]
W3 and Q3: Negative sampling and annotation sensitivity.
We thank the reviewer for this valuable suggestion.
Negative sampling. Following it and Reviewer 4FpS's related comment we rebuilt the ID negatives with the AI-Bind protocol, taking each decoy at bipartite network distance >= 7 for training and >= 11 for evaluation while leaving the positives and the partition unchanged, and re-evaluated the baselines. The results are shown below.
Table R4. Baselines under AI-Bind negative sampling on the ID split.
| Task | Binary binding prediction | Binding-site localisation | ||||||
|---|---|---|---|---|---|---|---|---|
| Model | AUROC | AUPRC | MCC | F1 | BRHR@1 | Precision@5 | Recall@5 | MAP |
| MolTrans | 77.5 | 32.4 | 21.7 | 17.2 | 10.3 | 4.7 | 1.3 | 9.5 |
| TransformerCPI | 76.0 | 32.8 | 21.0 | 16.0 | 9.8 | 4.6 | 1.4 | 8.9 |
| HyperAttentionDTI | 79.5 | 35.1 | 20.2 | 18.1 | 11.4 | 4.5 | 2.0 | 9.7 |
| PerceiverCPI | 78.1 | 37.9 | 21.8 | 18.6 | 9.5 | 4.1 | 1.4 | 9.0 |
| CAT-DTI | 80.3 | 36.9 | 23.4 | 18.5 | 12.8 | 4.7 | 1.2 | 8.6 |
| DrugBAN | 81.7 | 38.0 | 25.9 | 20.3 | 15.6 | 7.2 | 1.9 | 10.1 |
| GraphBAN | 82.3 | 38.5 | 25.1 | 20.4 | 16.4 | 6.5 | 1.9 | 9.7 |
| FusionDTI | 84.0 | 40.9 | 26.9 | 23.4 | 19.2 | 7.4 | 2.4 | 11.0 |
Binary binding prediction becomes harder under the distance-based negatives while binding-site localization is largely unaffected. We will cite AI-Bind in Sec. 1.2, describe both negative constructions in Sec. 2.1, and release the distance-based negatives alongside the current ones.
Annotation sensitivity. We thank the reviewer for this suggestion. We adopt published tool-specific criteria: PLIP (Salentin et al., 2015; Schake et al., 2025) identifies hydrogen bonds, salt bridges, π–π stacking, cation–π interactions and hydrophobic contacts, while GetContacts provides van der Waals annotations. These cover six non-covalent interaction categories discussed by Cerny and Hobza (2007).
InteractBind assigns distance-decay weights to detected contacts, but these do not eliminate the tools’ geometric thresholds. Our residue-level localisation metrics use reference binding-residue sets rather than numerical contact weights; changes to weights alone therefore do not affect evaluation when set membership remains unchanged. However, alternative tools or cutoffs may alter these sets and consequently the scores. The binary-only baselines do not use these annotations during training, whereas the map-supervised baseline does. We will clarify this distinction in Appendix B.2 and acknowledge sensitivity to alternative annotation settings as a limitation.
References
-
Non-covalent interactions in biomacromolecules. Physical Chemistry Chemical Physics, 9(39):5291-5303, 2007.
-
GetContacts: Interaction analysis for molecular structures and ensembles, 2018.
-
PLIP: fully automated protein-ligand interaction profiler. Nucleic Acids Research, 43(W1):W443-W447, 2015.
-
PLIP 2025: introducing protein-protein interactions to the protein-ligand interaction profiler. Nucleic Acids Research, 53(W1):W463-W465, 2025.
Dear Reviewer qoYe,
Thank you again for your thoughtful review and the time you have devoted to our paper. As the discussion period nears its end, we would sincerely appreciate it if you could take a look and, if you find our responses satisfactory, kindly consider reflecting this in your evaluation. We would also be happy to address any remaining questions.
Kind regards,
The Authors
Response to Reviewer qoYe [1]
We thank the reviewer for the careful reading and for the positive assessment of InteractBind's residue-level annotations, task formulation and similarity-controlled OOD splits. We respond to the three concerns in turn, and answer each question within the corresponding weakness.
W1 and Q2: No baseline trained with a localization objective.
We thank the reviewer for this valuable suggestion. We agree, and have added a supervised localization baseline, a deliberately simple model that encodes the protein with ESM-2 and the ligand with SELFormer and joins them through a cross-attention fusion module, trained with a Kullback-Leibler term that aligns the predicted interaction map with the annotated one alongside the binary objective, and we evaluate it against FusionDTI, the strongest binary-only baseline in Table 2, on the ID split and on the four composition-matched OOD splits. The results are shown below.
Table R5. Supervised localization baseline against the strongest binary-only baseline, on the ID split and on the OOD splits averaged over the four thresholds (%).
The supervised baseline improves all four localisation metrics on both ID and OOD splits while maintaining comparable binary prediction performance. These results support the value of explicit localisation supervision and demonstrate that InteractBind’s annotations are learnable. We have added this baseline to Table 2 and Appendix C and revised Sections 4.1 and 5 accordingly.
W2 and Q1: Clarification of ID splitting and train–test overlap.
We thank the reviewer for this valuable question. The ID folds are partitioned at the unique protein–ligand pair level, with each pair defined by its protein amino acid sequence and ligand SMILES string. All records sharing the same sequence–SMILES pair are assigned to a single fold. We verified that no such pair occurs in both training and test sets within any fold, although individual protein sequences and ligand SMILES strings may appear in both sets.
Table R6. Train–test overlap in the ID setting (%).
The ID setting evaluates held-out pairs within the same distribution, rather than generalisation to unseen proteins or ligands. Generalisation is assessed separately using the protein- and ligand-similarity-controlled OOD splits. We will clarify the pair definition and splitting protocol in Section 3 and report the overlap statistics in Appendix A.