acceptodds
Under review as a conference paper at ICLR 2027

A Large-Scale Dataset and Benchmark: Do Protein–Ligand Models Learn Binding Sites or Just Binding Likelihood?

Abstract

Protein-ligand modeling underpins computational drug discovery and molecular design. Existing protein-ligand benchmarks typically evaluate whether a protein and ligand interact and how strongly they bind, through tasks such as binary binding prediction and affinity regression. However, these evaluations provide limited evidence of whether models can localize binding sites or identify the non-covalent interactions underlying molecular recognition. To address this gap, we introduce **InteractBind**, a large-scale protein-ligand dataset comprising approximately 100k protein-ligand pairs, together with a benchmark for fine-grained evaluation. The core fine-grained task is that of binding-site localization, which uses protein-residue and ligand-atom interaction maps spanning six major types of non-covalent interactions to assess whether model-derived interaction maps localize binding sites. InteractBind further includes protein similarity-controlled splits to support realistic generalization assessment. Using InteractBind, we evaluate eight existing sequence-based and interaction-aware models, assessing binary binding prediction and binding-site localization. Results reveal limited binding-site localization despite strong binary binding prediction, with marked variation across non-covalent interaction types. Overall, InteractBind establishes a benchmark paradigm that encourages the development of more interpretable and physically grounded protein-ligand models.

Primary area:
datasets and benchmarks
Keywords:
Protein–Ligand Binding, Binding-site Localization, Non-covalent Interactions

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_qoYeOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces InteractBind, a large-scale protein–ligand dataset (~100k pairs derived from the PDB) and benchmark focused on residue-level binding-site localization. It supplies sequence-level interaction maps spanning six major non-covalent interaction types and uses these annotations to test whether model-derived interaction maps recover the residues involved in binding. The benchmark comprises an in-distribution (ID) split together with protein-similarity-controlled out-of-distribution (OOD) splits. Eight existing sequence-based and interaction-aware models are evaluated on both binary binding prediction and localization. The central empirical finding is that models can achieve strong binary performance while still exhibiting limited localization ability.

Critical strengths
  1. InteractBind supplies a form of residue-level, interaction-type-specific supervision that is largely absent from existing binary-binding or affinity-focused protein–ligand benchmarks, thereby enabling finer-grained evaluation of molecular recognition.
  2. The task formulation is clear and well-motivated: binary prediction and residue-level localization are assessed separately, with localization metrics that cover both top-ranked residues and overall ranking quality.
  3. The protein-similarity-controlled OOD splits provide a more realistic stress test of generalization to less familiar targets than random-split accuracy alone; the main result—that binary success does not imply reliable localization—is simple, reproducible, and relevant to the community.
Critical weaknesses
  1. All reported baselines are existing models trained primarily for binary binding prediction (or related pair-level objectives). They are not trained with a localization objective on the residue-level annotations. Consequently, the low localization scores may largely reflect the absence of direct supervision rather than an inherent inability of the architectures to learn binding sites. Because the benchmark is intended to guide future model design, the lack of at least one supervised localization baseline limits the strength of the claims.
  2. The ID split treats curated PDB pairs as positives and generates negatives by random re-pairing. It is not stated whether the folds are separated at the protein, ligand, or complex level. The same protein (or a close homolog) can therefore appear with different ligands in both training and test data, potentially inflating both binary and localization numbers. Explicit entity-level leakage statistics and/or stricter entity-separated splits would make the standard results easier to interpret.
  3. Negative sampling and interaction-label construction may introduce shortcuts or noise. Random re-pairing can yield relatively easy negatives, while the rule-based derivation of non-covalent contacts from crystal structures is sensitive to tool settings, distance cutoffs, and geometric criteria. Harder negatives, inter-annotator or tool-agreement checks, and cutoff-sensitivity analyses would strengthen the reliability of the labels and the trustworthiness of cross-model comparisons.
Questions for authors
  1. Are the standard (ID) evaluation folds separated by protein, ligand, or complex, or only by pair? What are the rates of protein- and ligand-level overlap between train and test?
  2. Can results for a model trained directly on the residue-level (and/or interaction-type-specific) annotations be added as a supervised localization baseline?
  3. How do binary and localization metrics change under hard-negative sampling and under alternative interaction-annotation settings (different tools or cutoffs)?
3 comments
AuthorsOpenReview ↗

Response to Reviewer qoYe [1]

We thank the reviewer for the careful reading and for the positive assessment of InteractBind's residue-level annotations, task formulation and similarity-controlled OOD splits. We respond to the three concerns in turn, and answer each question within the corresponding weakness.

W1 and Q2: No baseline trained with a localization objective.

We thank the reviewer for this valuable suggestion. We agree, and have added a supervised localization baseline, a deliberately simple model that encodes the protein with ESM-2 and the ligand with SELFormer and joins them through a cross-attention fusion module, trained with a Kullback-Leibler term that aligns the predicted interaction map with the annotated one alongside the binary objective, and we evaluate it against FusionDTI, the strongest binary-only baseline in Table 2, on the ID split and on the four composition-matched OOD splits. The results are shown below.

Table R5. Supervised localization baseline against the strongest binary-only baseline, on the ID split and on the OOD splits averaged over the four thresholds (%).

SplitModelAUROCAUPRCMCCF1BRHR@1Precision@5Recall@5MAP
IDFusionDTI (binary only)85.242.428.324.619.47.52.311.1
IDOurs (binary + map KL)85.741.528.124.943.410.94.519.5
OODFusionDTI (binary only)87.086.858.879.215.75.21.910.3
OODOurs (binary + map KL)87.485.658.178.939.08.23.917.2

The supervised baseline improves all four localisation metrics on both ID and OOD splits while maintaining comparable binary prediction performance. These results support the value of explicit localisation supervision and demonstrate that InteractBind’s annotations are learnable. We have added this baseline to Table 2 and Appendix C and revised Sections 4.1 and 5 accordingly.


 

W2 and Q1: Clarification of ID splitting and train–test overlap.

We thank the reviewer for this valuable question. The ID folds are partitioned at the unique protein–ligand pair level, with each pair defined by its protein amino acid sequence and ligand SMILES string. All records sharing the same sequence–SMILES pair are assigned to a single fold. We verified that no such pair occurs in both training and test sets within any fold, although individual protein sequences and ligand SMILES strings may appear in both sets.

Table R6. Train–test overlap in the ID setting (%).

LevelProtein–ligand pairProteinLigand
Overlap (%)0.048.940.0

The ID setting evaluates held-out pairs within the same distribution, rather than generalisation to unseen proteins or ligands. Generalisation is assessed separately using the protein- and ligand-similarity-controlled OOD splits. We will clarify the pair definition and splitting protocol in Section 3 and report the overlap statistics in Appendix A.

AuthorsOpenReview ↗

Response to Reviewer qoYe [2]

W3 and Q3: Negative sampling and annotation sensitivity.

We thank the reviewer for this valuable suggestion.

Negative sampling. Following it and Reviewer 4FpS's related comment we rebuilt the ID negatives with the AI-Bind protocol, taking each decoy at bipartite network distance >= 7 for training and >= 11 for evaluation while leaving the positives and the partition unchanged, and re-evaluated the baselines. The results are shown below.

Table R4. Baselines under AI-Bind negative sampling on the ID split.

TaskBinary binding predictionBinding-site localisation
ModelAUROCAUPRCMCCF1BRHR@1Precision@5Recall@5MAP
MolTrans77.532.421.717.210.34.71.39.5
TransformerCPI76.032.821.016.09.84.61.48.9
HyperAttentionDTI79.535.120.218.111.44.52.09.7
PerceiverCPI78.137.921.818.69.54.11.49.0
CAT-DTI80.336.923.418.512.84.71.28.6
DrugBAN81.738.025.920.315.67.21.910.1
GraphBAN82.338.525.120.416.46.51.99.7
FusionDTI84.040.926.923.419.27.42.411.0

Binary binding prediction becomes harder under the distance-based negatives while binding-site localization is largely unaffected. We will cite AI-Bind in Sec. 1.2, describe both negative constructions in Sec. 2.1, and release the distance-based negatives alongside the current ones.

Annotation sensitivity. We thank the reviewer for this suggestion. We adopt published tool-specific criteria: PLIP (Salentin et al., 2015; Schake et al., 2025) identifies hydrogen bonds, salt bridges, π–π stacking, cation–π interactions and hydrophobic contacts, while GetContacts provides van der Waals annotations. These cover six non-covalent interaction categories discussed by Cerny and Hobza (2007).

InteractBind assigns distance-decay weights to detected contacts, but these do not eliminate the tools’ geometric thresholds. Our residue-level localisation metrics use reference binding-residue sets rather than numerical contact weights; changes to weights alone therefore do not affect evaluation when set membership remains unchanged. However, alternative tools or cutoffs may alter these sets and consequently the scores. The binary-only baselines do not use these annotations during training, whereas the map-supervised baseline does. We will clarify this distinction in Appendix B.2 and acknowledge sensitivity to alternative annotation settings as a limitation.

References

  1. Non-covalent interactions in biomacromolecules. Physical Chemistry Chemical Physics, 9(39):5291-5303, 2007.

  2. GetContacts: Interaction analysis for molecular structures and ensembles, 2018.

  3. PLIP: fully automated protein-ligand interaction profiler. Nucleic Acids Research, 43(W1):W443-W447, 2015.

  4. PLIP 2025: introducing protein-protein interactions to the protein-ligand interaction profiler. Nucleic Acids Research, 53(W1):W463-W465, 2025.

Dear Reviewer qoYe,

Thank you again for your thoughtful review and the time you have devoted to our paper. As the discussion period nears its end, we would sincerely appreciate it if you could take a look and, if you find our responses satisfactory, kindly consider reflecting this in your evaluation. We would also be happy to address any remaining questions.

Kind regards,
The Authors

Reviewer_4FpSOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Authors propose InteractBind, a new benchmark dataset with ~100k pairs that not only looks at whether or not a protein and ligand bind, but also where they bind, focussing on binding-site localization as the primary evaluation task. The datasets also contains an in-distribution (ID) split and protein similarity-controlled out-of-distribution (OOD) split.

Critical strengths
  1. The dataset seems quite comprehensive with each sample containing protein and ligand sequence representations, binary binding labels, affinity values, binding-site annotations, and weighted interaction maps for six non-covalent interactions and an overall interaction map
  2. Table 1 is a good comparison briefly talking about the holes this paper covers.
  3. Good analysis in terms of number of baseline models as well as the discussion in terms of what types of binding sites are harder to detect and why.
Critical weaknesses

Not critical, but a standard, non-critical weakness:

  1. Negatives are randomly sampled. This could be improved by hard negative sampling as also commonly done in the literature (see for example [1] that is already mentioned). Although degree preserving negatives are a nice touch.

(1) Chatterjee, A., Walters, R., Shafi, Z. et al. Improving the generalizability of protein-ligand binding predictions with Al-Bind. Nat Commun 14, 1989 (2023). https://doi.org/10.1038/s41467-023-37572-z

Questions for authors
3 comments
AuthorsOpenReview ↗

Response to Reviewer 4FpS

Thank you for your positive assessment and constructive feedback. We appreciate your recognition of the comprehensiveness of InteractBind's annotations, the dataset comparison, and the breadth of our baseline evaluations and localization analyses.

Following your suggestion, we applied the AI-Bind negative-sampling protocol (Chatterjee et al., 2023) to the ID split. The positives and the partition are unchanged, and the only difference is how each decoy is drawn. Instead of re-pairing at random from the empirical frequency distribution, it is taken at bipartite network distance >= 7 for training and >= 11 for evaluation. The 1:10 ratio is retained throughout.

Table R4. Baselines under AI-Bind negative sampling on the ID split.

TaskBinary binding predictionBinding-site localisation
ModelAUROCAUPRCMCCF1BRHR@1Precision@5Recall@5MAP
MolTrans77.532.421.717.210.34.71.39.5
TransformerCPI76.032.821.016.09.84.61.48.9
HyperAttentionDTI79.535.120.218.111.44.52.09.7
PerceiverCPI78.137.921.818.69.54.11.49.0
CAT-DTI80.336.923.418.512.84.71.28.6
DrugBAN81.738.025.920.315.67.21.910.1
GraphBAN82.338.525.120.416.46.51.99.7
FusionDTI84.040.926.923.419.27.42.411.0

The results indicate that negatives drawn by network distance make binary binding prediction harder than random re-pairing, while having only a minor effect on binding-site localisation. We will cite AI-Bind in the revised manuscript and add this negative set to InteractBind alongside the current one, so that both constructions are available to users of the benchmark.

Thank you again for your thoughtful review and encouraging comments.

Ah interesting! Thank you for running this, it's quite informative that it doesnt help as much.

Dear Reviewer 4FpS,

Thank you again for your valuable suggestion. This was a worthwhile experiment, providing an additional perspective on how negative sampling affects model evaluation.

Reviewer_1sxPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents the InteractBind dataset, which contains approximately 100,000 protein-ligand pairs along with annotations of their binding sites and non-covalent interactions. By evaluating eight existing models, the authors find that good binding prediction performance does not mean the model can accurately localize binding sites. This dataset and evaluation framework support the finer-grained assessment of protein-ligand models.

Critical strengths

The research question carries practical significance: it focuses on whether the model can accurately localize binding sites, compensates for the limitation of only evaluating binding prediction performance, and helps verify the reliability of model explanations.

The dataset features comprehensive annotations: it contains approximately 100,000 protein-ligand pairs with annotations for six types of non-covalent interactions. A test split with controlled protein similarity is adopted to support fine-grained evaluation and generalization analysis

Critical weaknesses
  1. Evidence for conclusions on fine-grained interactions remains limited. BRHR@5 reported by interaction type on Page 8 only checks whether predicted protein residues hit the corresponding annotated sets, without validating specific ligand atoms or predicted interaction types. Therefore, this experiment supports analyzing the binding residue localization capability by interaction type, but provides limited support for the stronger conclusion of "recognizing non-covalent interaction patterns".

  2. The robustness of localization results to interaction graph extraction and residue mapping approaches remains unclear. Appendix C.1 adopts different signal extraction methods for different models. For instance, PerceiverCPI uses gradient saliency, while other models mainly employ attention or interaction matrices; some models also map multi-residue tokens to central residues. The paper does not examine the impacts of these choices on precise residue hit rates and model rankings, making it difficult to distinguish differences in model capability from effects introduced by evaluation processing pipelines.

  3. Confounding factors exist in interpreting performance disparities across different OOD splits. Table 3 on Page 15 shows that the numbers of proteins and ligands vary under different protein similarity thresholds. The performance trends observed on Page 8 may thus be affected by both similarity and sample composition; current experiments have not fully isolated the effect of protein similarity itself.

Questions for authors

see the weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1sxP

We thank the reviewer for the thoughtful and detailed reading of our work, and respond to the three points in turn.

W1: Evidence for conclusions on fine-grained interactions.

We thank the reviewer for raising this point. To clarify, our primary fine-grained task is binding-site localization, rather than interaction-type prediction. The interaction type is provided by the ground-truth annotation, and the per-type analysis evaluates whether model-derived interaction maps localize residues involved in each annotated interaction category.

We have revised Sec. 2 and Sec. 4.3 to state explicitly that the per-type results decompose binding-residue localization by annotated interaction category, rather than evaluate direct prediction of non-covalent interaction types. Direct residue–atom interaction-type prediction represents a stronger task, and we view InteractBind as providing the annotations needed to support this direction in future work.


 

W2: Robustness to interaction-map extraction and residue mapping.

We thank the reviewer for this valuable suggestion. We agree that the extraction and alignment choices are design decisions whose effect is worth quantifying. We extract from each model the signal its own authors designate as the residue-atom interaction map, but the native protein-side unit differs (BPE token, k-mer, convolutional receptive field, or residue), so every map is projected onto a common residue axis before any metric is computed.

We therefore re-scored all eight models on the ID split under two alternative projections, holding the trained checkpoints fixed so that only the projection changes.

Table R2. Effect of the unit-to-residue projection on the ID results (BRHR@1, %).

ModelReported (centre residue)Start of spanEnd of span
MolTrans10.59.810.1
TransformerCPI9.79.29.5
HyperAttentionDTI11.711.011.3
PerceiverCPI9.69.49.3
CAT-DTI12.611.912.2
DrugBAN15.815.215.5
GraphBAN16.315.716.0
FusionDTI19.419.419.4
Spearman’s ρ versus reported ranking1.0000.9761.000
Largest absolute change in BRHR@1 (pp)0.00.70.4

The tested projections produce similar overall rankings, with Spearman’s ρ of 0.976 and 1.000 and maximum absolute changes of 0.7 and 0.4 percentage points, respectively. The start-of-span projection reverses the order of two closely performing models. We will add these results to Appendix C.1, together with explicit descriptions of the model-specific extraction procedures and unit-to-residue mapping rules.


 

W3: Confounding factors across OOD splits.

We thank the reviewer for this valuable suggestion. Following it, we rebuilt the four protein similarity-controlled splits so that the number of pairs (4,800), proteins (2,800) and ligands (3,600) is identical across thresholds, leaving train-test sequence identity as the only variable. The results are shown below.

Table R3. Performance on the composition-matched OOD splits (%).

ModelAUROC 25 / 28 / 31 / 33BRHR@5 25 / 28 / 31 / 33
MolTrans72.8 / 74.9 / 76.2 / 77.519.3 / 20.0 / 20.6 / 20.3
TransformerCPI73.5 / 75.2 / 77.1 / 78.617.8 / 19.0 / 18.6 / 19.7
HyperAttentionDTI76.1 / 77.8 / 79.0 / 80.320.1 / 21.0 / 20.7 / 21.6
PerceiverCPI78.4 / 80.1 / 81.0 / 82.718.2 / 19.1 / 18.7 / 19.3
CAT-DTI80.2 / 82.4 / 84.0 / 85.319.1 / 20.4 / 20.0 / 20.8
DrugBAN81.0 / 83.1 / 84.6 / 86.025.6 / 26.8 / 26.3 / 27.0
GraphBAN83.1 / 85.0 / 86.8 / 88.025.9 / 26.4 / 27.2 / 26.8
FusionDTI84.0 / 86.3 / 88.1 / 89.426.8 / 27.7 / 27.4 / 28.2

These results are consistent with Sec. 4.4. Train-test protein similarity affects both binary binding prediction and binding-site localization, with performance improving as the similarity constraint becomes less stringent, and the effect remains stronger on binary prediction than on localization.

Dear Reviewer 1sxP,

Thank you again for your thoughtful review and the time you have devoted to our paper. As the discussion period nears its end, we would sincerely appreciate it if you could take a look and, if you find our responses satisfactory, kindly consider reflecting this in your evaluation. We would also be happy to address any remaining questions.

Kind regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.