acceptodds
Under review as a conference paper at ICLR 2027

PLI-CLIP: Learning Transferable Protein–Ligand Interaction Representations from Natural Language Supervision

Abstract

Predicting the 3D structure of protein–ligand complexes is central to structure-based drug discovery, and recent approaches employ protein language models (PLMs) to address this challenge. While PLMs provide powerful representations of proteins, they are inherently ligand-agnostic and do not capture explicit protein–ligand interaction semantics. To address this shortcoming, we exploit rich, text-like biochemical annotations produced by tools such as PLIP to learn ligand-aware protein representations. Specifically, we construct the Protein-Ligand Interaction Text Dataset (PLI-TD) by translating structured PLIP profiles into grounded, natural-language interaction descriptions. We then introduce PLI-CLIP, a CLIP-style contrastive framework that aligns protein features with PLI-TD to inject explicit interaction semantics. While these textual descriptions specify which interactions (e.g., hydrogen bonds, hydrophobic contacts) occur, they do not localize them. Therefore, PLI-CLIP is further trained to predict per-residue binding geometry, transforming geometry-blind representations into geometry-aware ones. Finally, these representations serve as a transferable module that integrates into existing structure prediction models without modifying their original architectures, eliminating the need for text annotations or interaction profiles at inference time. PLI-CLIP consistently improves docking accuracy across various recent downstream docking models, raising DiffDock's apo top-1 rate (24.4 → 30.3), NeuralPLexer's <2 Å fraction (36.6 → 40.2), and FlowDock's docking success rate (50.4 → 54.1).

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein–Ligand Interaction, Molecular Docking, Contrastive Learning, Protein Language Model, Multimodal Representation Learning, Structure-Based Drug Discovery

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_NbTuOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces PLI-CLIP, a framework for learning transferable, ligand-conditioned protein representations from interaction descriptions. The authors construct PLI-TD by converting PLIP-derived protein–ligand interaction records into textual descriptions, and jointly train a protein–ligand encoder using text–complex contrastive alignment and per-residue geometry prediction. The resulting features are injected into downstream docking models through a residual adapter, followed by backbone fine-tuning. Experiments on DiffDock, NeuralPLexer, and FlowDock show improvements in docking accuracy and physically valid pose prediction, without requiring interaction descriptions or bound poses at inference.

Critical strengths

A useful approach to transferring interaction supervision into docking representations. The framework uses interaction annotations available from resolved training complexes while avoiding their requirement at inference. Combining ligand conditioning, interaction-text alignment, and residue-level geometry prediction provides a coherent approach to enriching sequence-derived protein features. This addresses a relevant limitation of ligand-agnostic protein representations.

Positive transfer across multiple docking architectures. Improvements are reported for three distinct backbones rather than a single tailored model. Particularly encouraging results include DiffDock’s top-1 success rate increasing from 24.36% to 30.31% with ESMFold protein structures, and FlowDock’s relaxed, accurate-and-PB-valid fraction increasing from 50.4% to 54.1%. These results support the practical usefulness of the representation, while the inclusion of physical-validity metrics strengthens the evaluation beyond pose RMSD alone.

Critical weaknesses

The specific contribution of natural-language supervision remains insufficiently isolated. Comparing different textual renderings does not establish whether language provides an advantage over directly encoding or predicting the same structured PLIP interaction records. Moreover, Table 4 removes both an objective and its corresponding feature component, confounding supervision with the downstream feature configuration. These limitations weaken the attribution of the gains to the proposed language–geometry combination, although they do not negate the utility of the complete framework.

The robustness of the reported improvements needs clearer support. The paper does not report variability across runs or confidence intervals, which matters for interpreting relatively modest gains in stochastic docking models. While excluding evaluation complexes from pretraining is a positive precaution, exact-complex exclusion alone does not characterize generalization across similar proteins and ligands. More explicit reporting of baseline fine-tuning conditions and train–test similarity would strengthen confidence in the comparisons.

Questions for authors

Clarification: Were the baselines subjected to the same additional fine-tuning schedule, EMA, and checkpoint-selection procedure as the PLI-CLIP-enhanced models? Please specify how the training and selection budgets were matched.

Evidence request: Do you have a comparison against directly using the same structured PLIP records as supervision? An objective ablation that preserves the downstream feature dimensions would also help distinguish the effects of supervision from changes in the injected representation.

Clarification: Can you provide uncertainty estimates for the main improvements, and characterize protein-sequence and ligand-scaffold similarity between training and evaluation sets? These would help assess whether the gains are stable and how broadly they generalize.

Related-work suggestion, separate from the assessment-changing questions: The discussion of structure-aware protein pretraining could be broadened to include Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains (ICML 2024), which studies joint residue- and atom-level representations through span masking. Discussing this work would help position PLI-CLIP relative to complementary approaches for enriching protein representations with structural information, while clarifying its distinct emphasis on ligand-conditioned interaction supervision

5 comments
AuthorsOpenReview ↗

Response to Reviewer NbTu

To Reviewer NbTu

We thank the reviewer for the careful reading and thoughtful feedback. We are currently running experiments to address these concerns, and we report those already complete so that the discussion can begin. We will respond to the remaining points (Question 2 and 3) within this week.

[Q1&W2] Matched fine-tuning schedule, EMA and checkpoint selection.

In the submitted version, the baselines were evaluated from the officially released weights of each backbone, without the additional Stage-2 fine-tuning or EMA applied to the PLI-CLIP models. We agree that this left the comparison unmatched, and we have now added the matched control for all three backbones. Each baseline is fine-tuned for the same 10 epochs at the same learning rate (1e-4) on the backbone's own training data, evaluated with the same EMA of the fine-tuned weights, and selected by the same rule, so the only difference is the presence of the pretrained interaction representation. The Fine-tuning column marks whether Stage-2 fine-tuning was applied.

DiffDockfine-tuningHolo Top-1 %<2Å ↑Holo Top-1 Med ↓Holo Top-5 %<2Å ↑Holo Top-5 Med ↓Apo Top-1 %<2Å ↑Apo Top-1 Med ↓Apo Top-5 %<2Å ↑Apo Top-5 Med ↓
DiffDockX39.892.9347.012.1624.366.0135.413.66
DiffDockO39.892.8347.862.1223.515.7836.543.70
DiffDock+PLI-CLIPO41.882.6949.572.0230.314.7238.813.22
NeuralPLexerfine-tuningTop-1 25th ↓Top-1 50th ↓Top-1 %<5Å ↑Top-1 %<2Å ↑Top-5 25th ↓Top-5 50th ↓Top-5 %<5Å ↑Top-5 %<2Å ↑
NeuralPLexerX1.653.2165.136.61.553.0975.146.0
NeuralPLexerO1.683.2364.936.71.573.1175.545.6
NeuralPLexer+PLI-CLIPO1.302.6269.540.21.052.1477.347.9
FlowDockfine-tuningHolo RMSD<2Å ↑Holo RMSD<2Å & PB-Valid ↑Apo RMSD<2Å ↑Apo RMSD<2Å & PB-Valid ↑
FlowDockX71.434.071.050.4
FlowDockO71.534.270.849.9
FlowDock+PLI-CLIPO72.238.472.254.1

As the three results above show, fine-tuning on its own does not account for the improvements. Performance is essentially unchanged across all three backbones, and on DiffDock with ESMFold structures top-1 accuracy in fact drops from 24.36 to 23.51. NeuralPLexer moves only from 36.6 to 36.7, and FlowDock from 34.0 to 34.2 on the accurate-and-PB-valid fraction. With PLI-CLIP, by contrast, all three backbones improve consistently under the identical schedule.

[Q4] Related work: span-mask bi-level protein pretraining (ICML 2024).

We have added a discussion of this work to Section 2, marked in blue. Its supervision is intrinsic to the protein chain, which makes it complementary to our approach, where the supervision is conditioned on a specific bound ligand and derived from observed interaction records, so it was indeed worth including. We thank the reviewer for the suggestion.

AuthorsOpenReview ↗

Response to Reviewer NbTu (2)

To Reviewer NbTu

We have now completed the remaining experiments and address the last two points below. Thank you for your patience.


[Q2 & W1] Structured-record supervision, and an ablation at fixed feature dimension

  • Structured PLIP records as supervision.

We trained a variant on DiffDock that encodes the same PLIP records directly as interaction vectors, bypassing the text rendering and the text encoder, and aligns the protein–ligand representation to those vectors under the identical contrastive objective. The target is a 140-dimensional vector counting interactions for each of the twenty residue types paired with each of seven categories, which refine the paper’s six PLIP types by splitting hydrogen bonds on the donor side. It replaces the sentence embedding as the InfoNCE target, retaining the interaction content the captions verbalise. The encoder, the Stage-2 adapter, and the fine-tuning schedule are unchanged, so the only variable is whether the record passes through language.

ModelCLIP TrainingHolo Top-1 %<2Å ↑Holo Top-1 Med ↓Holo Top-5 %<2Å ↑Holo Top-5 Med ↓Apo Top-1 %<2Å ↑Apo Top-1 Med ↓Apo Top-5 %<2Å ↑Apo Top-5 Med ↓
DiffDock–39.892.9347.012.1624.366.0135.413.66
DiffDock+PLI-CLIPvector encoding40.172.8548.152.0728.905.0137.963.14
DiffDock+PLI-CLIPPLM41.882.6949.572.0230.314.7238.813.22

Encoding the records directly already improves over the backbone, most visibly on ESMFold structures (24.36 to 28.90), so the interaction supervision transfers and is not specific to language. Language supervision is better still on seven of eight columns, reaching 41.88 on holo and 30.31 on apo top-1; the exception is the apo top-5 median (3.14 against 3.22). We therefore do not claim language as the sole source of the gain, only that interaction records seen at training time transfer into a representation needing no annotation at inference, and that text is the more effective vehicle.

  • Ablation at fixed feature dimension.

We agree that Table 4 confounded the supervision with the injected feature, and we have rerun it on NeuralPLexer with the feature dimension held fixed: the ablated component is retained and zero-filled, so the input dimension of WinjW_{inj} and the Stage-2 schedule are unchanged, and the only variable is which component carries information.

φTop-1 25th ↓Top-1 50th ↓Top-1 %<5Å ↑Top-1 %<2Å ↑Top-5 25th ↓Top-5 50th ↓Top-5 %<5Å ↑Top-5 %<2Å ↑
[none]1.653.2165.136.61.553.0975.146.0
[0; e\mathbf{e}]1.503.1766.036.71.402.7775.945.9
[zc\mathbf{z}_c; 0]1.472.9967.237.91.132.4476.446.0
[zc\mathbf{z}_c; e\mathbf{e}]1.302.6269.540.21.052.1477.347.9

Each component alone gives a modest gain and the full representation is best on all eight columns: the text component leads in isolation (37.9 against 36.7 on top-1 RMSD<2Å), but neither approaches the 40.2 obtained by combining them. The pattern reported in the submitted Table 4 therefore does not depend on the coupling.

Both tables will be added to the revision.

[Q3] Uncertainty estimates and train–test similarity

  • Variability across runs.

To address the reviewer NbTu’s concern, we repeated Stage-2 training with three random seeds under an otherwise identical configuration on all three backbones (DiffDock, NeuralPLexer, FlowDock).

DiffDockRunsHolo Top-1 %<2Å ↑Holo Top-1 Med ↓Holo Top-5 %<2Å ↑Holo Top-5 Med ↓Apo Top-1 %<2Å ↑Apo Top-1 Med ↓Apo Top-5 %<2Å ↑Apo Top-5 Med ↓
DiffDock–39.892.9347.012.1624.366.0135.413.66
DiffDock + PLI-CLIP141.882.6949.572.0230.314.7238.813.22
DiffDock + PLI-CLIP342.07 ± 0.112.77 ± 0.0148.91 ± 1.322.04 ± 0.00128.80 ± 1.794.75 ± 0.00739.47 ± 0.833.06 ± 0.02
NeuralPLexerRunsTop-1 25th ↓Top-1 50th ↓Top-1 %<5Å ↑Top-1 %<2Å ↑Top-5 25th ↓Top-5 50th ↓Top-5 %<5Å ↑Top-5 %<2Å ↑
NeuralPLexer–1.653.2165.136.61.553.0975.146.0
NeuralPLexer + PLI-CLIP11.302.6269.540.21.052.1477.347.9
NeuralPLexer + PLI-CLIP31.36 ± 0.072.61 ± 0.0469.2 ± 0.340.5 ± 0.31.09 ± 0.042.19 ± 0.1576.9 ± 0.447.2 ± 0.5
FlowDockRunsHolo RMSD<2Å ↑Holo RMSD<2Å & PB-Valid ↑Apo RMSD<2Å ↑Apo RMSD<2Å & PB-Valid ↑
FlowDock–71.434.071.050.4
FlowDock + PLI-CLIP172.238.472.254.1
FlowDock + PLI-CLIP372.6 ± 0.5138.2 ± 0.3971.9 ± 0.3753.0 ± 1.33

Across three seeds the improvements hold on every backbone. NeuralPLexer is the most stable, with standard deviations of 0.3–0.5 points on the below-threshold metrics and a mean of 40.5 against the backbone’s 36.6. FlowDock varies by 0.4–1.3 points and stays above the backbone throughout. DiffDock is the widest, with apo top-1 at 28.80 ± 1.79 against the single run’s 30.31, but the backbone reaches only 24.36 on the same evaluation, so the margin exceeds the spread. We will report mean and standard deviation in the revision.

  • Train–test similarity.

For every test complex we measured the maximum protein sequence identity with MMseqs2 [1] and the maximum ligand Tanimoto similarity over Morgan fingerprints [2] against PLI-TD, and separately against each backbone's own training set, together with Bemis–Murcko scaffold matches [3].

Test set vs PLI-TDMax seq id > 30%Max seq id = 100%Max Tanimoto > 0.7Scaffold match
PDBBind2020 (NeuralPLexer)74.4%43.0%29.2%36.1%
PoseBusters (FlowDock)54.9%16.6%27.9%43.2%
Test set vs training setMax seq id > 30%Max seq id = 100%Max Tanimoto > 0.7Scaffold match
PDBBind2020 (NeuralPLexer)84.8%39.9%20.7%33.1%
PoseBusters (FlowDock)59.1%21.1%34.4%47.4%

Residual similarity is considerable. On PDBBind2020, 43.0% of test complexes share an identical protein sequence with some PLI-TD entry under a different PDB code; the corresponding figure against the backbone's own training data is 39.9%, so PLI-TD adds little overlap beyond what the benchmark already carries. PoseBusters, drawn from later depositions, is markedly cleaner at 16.6%. We report these figures as they are. The comparison locates where the overlap comes from, but it does not tell us what that means for generalisation.

To address the latter, we removed the overlapping complexes and re-evaluated NeuralPLexer and FlowDock on the remaining disjoint subsets, filtering once by sequence identity and once by ligand similarity. The models are unchanged and only the evaluation set differs.

Reviewer_7JThOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper tries to capture explicit protein–ligand interaction semantics with language models. A text dataset is constructed to transform profile interactions into interaction descriptions. CLIP model has been integrated. Docking accuracy has been reported.

Critical strengths
  1. The overall framework is easy and straightforward.
  2. The abstract is well written. The writing has been well polished.
  3. A dataset is constructed, and the author promises to release it.
Critical weaknesses
  1. To me, the paper creates a meaningless problem regarding semantics and transferability.
  2. The contribution is really limited. The paper basically integrates the heavy CLIP model with PLMs. Even for the semantics, DrugCLIP and BindCLIP have already integrated CLIP into drug discovery.
  3. Second, it is really confusing why every research community introduces language models to solve the problem. Does semantics matter for drug discovery or biology? I don't think there is similarity between natural language and biological mechanisms. Are there any specific drugs whose discovery is based on PLMs? The paper has not justified the applicability of the method.
  4. As the name "PLI-CLIP" indicates, the paper basically combines PLM with CLIP. Both are very heavy models. Compared with an LLM dataset, the dataset used is really small, which may result in overfitting.
  5. The paper needs to compare with traditional protein–ligand baselines such as RNNs and graph models.
  6. The applicability of the constructed dataset has not been fully verified.
  7. While the first claim is a dataset, the paper should be submitted to the benchmark track.
  8. The training time and inference time have not been reported and compared with baselines.
  9. Presentation: It is likely that Figures 2 and 3 were generated by AI.
  10. Experiments are not extensive. More datasets and justification of transferability should be included.
Questions for authors

NA

2 comments
AuthorsOpenReview ↗

Response to Reviewer 7JTh

To Reviewer 7JTh

We thank the reviewer for the time and feedback. We address each point in the order raised.


[W1, W3] Motivation

Our premise is not that proteins and natural language are similar systems. It is that interaction records, which exist only for resolved complexes, can serve as training-time supervision for a representation that requires no such records at inference. All three backbones we build on already draw their protein features from a language model, ESM-2, so the question we address is what those features lack, not whether language models belong in this setting. Section 5 reports that a linear probe for interaction type reaches an AUROC of 0.964 on our representation against 0.812 for the ESM-2 features it is built from, and the experiments in our responses to the other reviewers quantify the effect on docking accuracy. We make no claim about specific approved drugs; our claim is about pose prediction accuracy on standard benchmarks.

[W2] Relation to prior CLIP-style work in drug discovery

DrugCLIP aligns binding pockets and molecules, both encoded as structures, with no textual supervision, and is used for virtual screening by retrieval. Our alignment is between a ligand-conditioned protein representation and a natural-language rendering of observed interaction records, and the resulting representation is transferred into existing docking backbones rather than used for retrieval. We were not able to locate a paper under the name BindCLIP and would be glad to discuss it if the reviewer can supply a reference. We will expand the related-work discussion to position the method against these approaches explicitly.

[W4] Model capacity and dataset size

The large pretrained components are never trained: ESM-2, Uni-Mol and PubMedBERT are frozen throughout, and Stage-1 trains only the ligand-aware cross-attention module and the three projection heads. Stage-2 adds a residual adapter of 164,353 parameters, or 0.23% of the NeuralPLexer backbone, and finetunes it together with the backbone on that backbone's own original training data, with PLI-CLIP frozen. On overfitting, all evaluation complexes are removed from the pretraining pool, and design choices were selected on a held-out validation split of 1,185 complexes rather than on the benchmarks. The disjoint-subset results in our response to Reviewer 9gu8 further show the improvements persist after removing test complexes that share a sequence-identical protein or a highly similar ligand with the pretraining set.

[W5] Comparison with earlier protein–ligand methods

Graph-based predecessors are discussed in Section 2: EquiBind and TankBind are the one-shot graph regression models in this line, and the three backbones we build on, DiffDock, NeuralPLexer and FlowDock, supersede them and are the standard points of comparison in current work. Sequence-only recurrent models predict binding affinity rather than ligand pose, so they cannot be evaluated on the RMSD and physical-validity metrics used here.

[W6, W7] Scope of the dataset contribution

PLI-TD is one of three contributions, alongside the contrastive alignment framework and the transferable adapter; the paper's central result is transfer across three independently developed docking backbones, which is a methodological finding rather than a benchmarking one. That transfer is also what establishes the dataset's applicability, and the disjoint-subset results in our response to Reviewer 9gu8 [W3&Q3] show the improvements survive after sequence- and ligand-overlapping test complexes are removed.

[W8] Training and inference time

On a single RTX 4090 with NeuralPLexer (70.8M) as the backbone, PLI-CLIP adds 164,353 parameters (+0.23%) and 45.7 ms per complex at inference, almost all of it the Uni-Mol ligand encoder, plus 7.9 MiB of resident GPU memory. ESM-2 is not counted because the backbone already runs it, and PubMedBERT is used only in Stage-1. Stage-2 is 10 epochs on the backbone's own training data and raises peak training memory by 2,464 MiB, which comes from retained activations rather than from the added parameters. Stage-1 is paid once and is reused across all three backbones without retraining. Full table is in our response to Reviewer 9gu8 [W6].

[W9] Figures 2 and 3

Figures were assembled by the authors in PowerPoint, element by element. Generative tools were used only for isolated decorative elements within them; the layout, the content, and every panel were composed by us. We are glad to provide the source files if that would be useful.

[W10] Extent of the evaluation

The evaluation covers two benchmarks, PDBBind2020 and PoseBusters, across three docking backbones, and includes physical-validity metrics alongside pose RMSD. Rather than adding further benchmarks, we have used the rebuttal period to test the generalisation question behind this point directly, on sequence- and ligand-disjoint subsets of the existing benchmarks.

AuthorsOpenReview ↗

Follow-up before the close of the discussion period

Dear Reviewer 7JTh,

As the discussion period closes today, could you please let us know whether our rebuttal has been helpful, or whether any concerns remain? We would be glad to provide further clarification.

Thank you for the time devoted to our submission.

Sincerely, the authors

Reviewer_9gu8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PLI-CLIP, a framework for learning transferable protein-ligand interaction representations through natural language supervision. The authors construct PLI-TD by converting PLIP-derived interaction profiles from experimentally determined protein-ligand complexes into natural language descriptions. PLI-CLIP then combines contrastive learning between protein-ligand representations and interaction texts with a geometry prediction objective. The learned representations are injected into existing docking models through a residual adapter. Experiments on DiffDock, NeuralPLexer, and FlowDock show improvements in docking accuracy, particularly for apo protein structures.

Critical strengths

The paper addresses an important problem in structure-based drug discovery, which is learning ligand-aware protein representations that can improve downstream docking performance. The idea of leveraging protein-ligand interaction information during pretraining without requiring interaction annotations at inference time is reasonable. A strength of this work is that the proposed representations are evaluated on three different docking models, demonstrating their potential transferability. The authors also provide ablation studies on textual representations, protein language model sizes, and training objectives.

Critical weaknesses
  1. The advantage of natural language supervision over directly using structured interaction information is not established. PLI-TD is constructed by converting structured PLIP interaction profiles into natural language descriptions using predefined templates. These profiles already contain interaction types, participating residues, distances, angles, and other geometric information. It is therefore unclear whether converting these structured records into natural language provides additional benefits. Although the authors compare three textual representations in Table 3, all three are derived from the same structured information. A comparison with a model that directly encodes the structured PLIP features would be important to demonstrate the necessity and advantage of natural language supervision.

  2. The experimental comparisons do not sufficiently isolate the contribution of PLI-CLIP pretraining. In Stage 2, the authors introduce a residual adapter and fine-tune the downstream docking backbones. Therefore, the performance improvements in Table 1 may come from the pretrained interaction representations, the additional model components, or the additional fine-tuning. The authors should include a baseline using the same adapter architecture and fine-tuning procedure without PLI-CLIP pretraining. In particular, since the proposed protein-ligand encoder already combines ESM-2 and Uni-Mol features through cross-attention, it would be useful to evaluate whether ligand-aware representations without interaction-text pretraining can achieve similar improvements.

  3. The potential overlap between pretraining and evaluation data requires further investigation. The authors exclude the evaluation complexes from Stage-1 pretraining, which is appropriate. However, excluding identical complexes does not necessarily prevent highly similar proteins or ligands from appearing in both datasets. Since the pretraining supervision is derived from experimentally determined interaction geometries, such similarities may affect the evaluation of transferability. The authors should report protein sequence similarity and ligand structural similarity between the pretraining and evaluation sets. Evaluation on protein-disjoint or sequence-cluster-disjoint test sets would provide stronger evidence of generalization.

  4. The ablation studies do not fully separate the effects of different training objectives. In Table 4, removing a training objective also removes its corresponding representation component from the transferred features. Therefore, the experiments change both the training objectives and the feature representations. This makes it difficult to determine whether the improvements come from the proposed supervision objectives or the additional representation components.

  5. The experimental results lack statistical stability analysis. For example, the FlowDock RMSD < 2 Å rate without post-processing increases only from 71.4% to 72.2%. Since the downstream models involve stochastic training and sampling, reporting results across multiple random seeds would help establish the robustness of these improvements.

  6. The computational overhead is not sufficiently evaluated. PLI-CLIP introduces additional pretrained encoders, cross-attention, and a residual adapter. Although the authors describe the framework as transferable and easy to integrate, the additional inference time, GPU memory consumption, and training cost are not reported.

Questions for authors
  1. Can the authors compare natural language supervision with directly encoding the same structured PLIP interaction profiles?

  2. Can the authors provide a baseline with the same residual adapter and downstream fine-tuning procedure but without PLI-CLIP pretraining?

  3. Have the authors examined protein sequence and ligand structural similarities between the pretraining and evaluation datasets? How does the method perform on protein-disjoint evaluation sets?

  4. Can the authors clarify how the complex-level representation (z_i^c) is combined with the residue-level geometric representation (e_i), and how the contrastive objective encourages residue-specific interaction learning?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 9gu8

To Reviewer 9gu8

We thank the reviewer for the detailed comments. We are currently running experiments to address all six points, and we report the three that are already complete so that the discussion can begin. We will respond to the remaining points within this week.

[W1&Q1] Necessity of natural-language supervision over structured PLIP records.

To verify whether language-based contrastive alignment is necessary, we trained a variant on DiffDock that encodes the same PLIP records directly as interaction vectors, bypassing the text rendering and the text encoder entirely, and aligns the protein–ligand representation to those vectors under the identical contrastive objective. The alignment target is a 140-dimensional vector, with one entry for each of the twenty residue types paired with each of the seven PLIP interaction categories, holding the number of such interactions in the complex. It replaces the sentence embedding as the InfoNCE target, so the text encoder is dropped while the interaction content the captions verbalise is retained.

The three rows below are the DiffDock backbone, this vector-based variant, and our proposed method. The protein–ligand encoder, the Stage-2 residual adapter, and the fine-tuning schedule are identical throughout, so the only variable is whether the interaction record passes through language.

CLIP TrainingHolo Top-1 %<2Å ↑Holo Top-1 Med ↓Holo Top-5 %<2Å ↑Holo Top-5 Med ↓Apo Top-1 %<2Å ↑Apo Top-1 Med ↓Apo Top-5 %<2Å ↑Apo Top-5 Med ↓
DiffDockX39.892.9347.012.1624.366.0135.413.66
DiffDock-PLI-CLIPvector encoding40.172.8548.152.0728.905.0137.963.14
DiffDock-PLI-CLIPPLM41.882.6949.572.0230.314.7238.813.22

Encoding the records directly already improves over the backbone, most visibly on ESMFold structures, where top-1 accuracy rises from 24.36 to 28.90. The interaction supervision itself therefore transfers, and this is not specific to language. Language supervision is better still, on seven of the eight columns, reaching 41.88 on holo and 30.31 on apo top-1; the one exception is top-5 median on apo, where the structured variant is marginally ahead at 3.14 against 3.22. We do not claim that language is the sole source of the improvement, and the table shows that it is not. Our claim is the narrower one that interaction records observed at training time can be transferred into a protein representation requiring no interaction annotation at inference, and that rendering them as text is the more effective vehicle for doing so.

[W2&Q2] Isolating the contribution of PLI-CLIP pretraining.

Table 1 did not include the result of applying the same Stage-2 fine-tuning procedure to each backbone, so we have added it for DiffDock, NeuralPLexer, and FlowDock. The training budget, EMA, and checkpoint-selection rule are matched to those of the PLI-CLIP models, so the only difference is the presence of the pretrained interaction representation; the Fine-tuning column marks whether Stage-2 fine-tuning was applied.

DiffDockFine-tuningHolo Top-1 %<2Å ↑Holo Top-1 Med ↓Holo Top-5 %<2Å ↑Holo Top-5 Med ↓Apo Top-1 %<2Å ↑Apo Top-1 Med ↓Apo Top-5 %<2Å ↑Apo Top-5 Med ↓
DiffDockX39.892.9347.012.1624.366.0135.413.66
DiffDockO39.892.8347.862.1223.515.7836.543.70
DiffDock-PLI-CLIPO41.882.6949.572.0230.314.7238.813.22
NeuralPLexerFine-tuningTop-1 25th ↓Top-1 50th ↓Top-1 %<5Å ↑Top-1 %<2Å ↑Top-5 25th ↓Top-5 50th ↓Top-5 %<5Å ↑Top-5 %<2Å ↑
NeuralPLexerX1.653.2165.136.61.553.0975.146.0
NeuralPLexerO1.683.2364.936.71.573.1175.545.6
NeuralPLexer-PLI-CLIPO1.302.6269.540.21.052.1477.347.9
FlowDockFine-tuningHolo RMSD<2Å ↑Holo RMSD<2Å & PB-Valid ↑Apo RMSD<2Å ↑Apo RMSD<2Å & PB-Valid ↑
FlowDockX71.434.071.050.4
FlowDockO71.534.270.849.9
FlowDock-PLI-CLIPO72.238.472.254.1

As the three results above show, fine-tuning on its own does not account for the improvements. Performance is essentially unchanged across all three backbones, and on DiffDock with ESMFold structures top-1 accuracy in fact drops from 24.36 to 23.51. NeuralPLexer moves only from 36.6 to 36.7, and FlowDock from 34.0 to 34.2 on the accurate-and-PB-valid fraction. With PLI-CLIP, by contrast, all three backbones improve consistently under the identical schedule.

[W3&Q3] Train–evaluation overlap; protein-disjoint evaluation.

To examine whether excluding the exact evaluation complexes is sufficient, we directly quantified the overlap. For every test complex, we computed the maximum protein sequence identity using MMseqs2 [1] and the maximum ligand Tanimoto similarity over Morgan fingerprints [2] against the PLI-CLIP pretraining set (PLI-TD) and, separately, against each backbone’s own training set. We also report how often the Bemis–Murcko scaffold [3] of the test ligand appears in those sets.

Test set vs PLI-TDMax seq id > 30%Max seq id = 100%Max Tanimoto > 0.7Scaffold match
PDBBind2020 (NeuralPLexer)74.4%43.0%29.2%36.1%
PoseBusters (FlowDock)54.9%16.6%27.9%43.2%
Test set vs Training setMax seq id > 30%Max seq id = 100%Max Tanimoto > 0.7Scaffold match
PDBBind2020 (NeuralPLexer)84.8%39.9%20.7%33.1%
PoseBusters (FlowDock)59.1%21.1%34.4%47.4%

The overlap is substantial, and we observe that it is characteristic of PDBBind-based evaluation rather than specific to our pretraining set: 43.0% of the PDBBind2020 test complexes have a protein in PLI-TD with an identical sequence under a different PDB entry, and the comparable figure against the backbone’s own training set is 39.9%. The two sides are therefore at a similar level, and we believe PLI-TD does not introduce a source of overlap that the backbone’s own training data does not already contain. We report these numbers plainly rather than minimising them, and we do not treat this comparison as a defence on its own, since it shows only that the overlap is not of our making and does not establish that the gains transfer to unseen targets.

For a closer analysis, we re-evaluated NeuralPLexer and FlowDock on the disjoint subsets obtained by removing the overlapping complexes, once by sequence identity and once by ligand similarity. (The models are unchanged; only the evaluation subset differs.)

<NeuralPLexer, excluding test complexes with a sequence-identical protein in PLI-TD>

Top-1 25th ↓Top-1 50th ↓Top-1 %<5Å ↑Top-1 %<2Å ↑Top-5 25th ↓Top-5 50th ↓Top-5 %<5Å ↑Top-5 %<2Å ↑
NeuralPLexer1.953.8654.426.01.522.8467.633.3
NeuralPLexer + PLI-CLIP1.743.7055.427.51.473.0468.136.8

<NeuralPLexer, excluding test complexes with a ligand of Tanimoto > 0.7 in PLI-TD>

Top-1 25th ↓Top-1 50th ↓Top-1 %<5Å ↑Top-1 %<2Å ↑Top-5 25th ↓Top-5 50th ↓Top-5 %<5Å ↑Top-5 %<2Å ↑
NeuralPLexer1.583.8159.233.31.252.6572.540.4
NeuralPLexer + PLI-CLIP1.433.6258.835.71.172.7077.343.5

<FlowDock, excluding test complexes with a sequence-identical protein in PLI-TD>

Excluding sequence-identicalHolo RMSD<2Å ↑Holo RMSD<2Å & PB-Valid ↑Apo RMSD<2Å ↑Apo RMSD<2Å & PB-Valid ↑
FlowDock71.632.869.650.5
FlowDock + PLI-CLIP72.436.672.050.9

<FlowDock, excluding test complexes with a ligand of Tanimoto > 0.7 in PLI-TD>

Excluding Tanimoto > 0.7Holo RMSD<2Å ↑Holo RMSD<2Å & PB-Valid ↑Apo RMSD<2Å ↑Apo RMSD<2Å & PB-Valid ↑
FlowDock71.336.669.850.8
FlowDock + PLI-CLIP70.838.172.055.6

Absolute accuracy falls for both the backbone and our model once the overlapping complexes are removed, confirming that these subsets are genuinely harder. The relative picture is nonetheless preserved: PLI-CLIP improves the backbone on thirteen of the sixteen columns above, with the largest margins on physical validity, raising the accurate-and-PB-valid fractions from 32.8 to 36.6 under the sequence filter and from 50.8 to 55.6 under the ligand filter. The three exceptions are small and scattered: NeuralPLexer’s top-5 median (3.04 vs 2.84) and top-1 RMSD<5Å (58.8 vs 59.2), and FlowDock’s holo RMSD<2Å (70.8 vs 71.3), where our model leads on the other three columns of that table. We take this as evidence that the learned representation is not simply recalling training complexes, though a purpose-built protein-family split would be a stronger test, which we will include in the final version.


[1] Steinegger, Martin, and Johannes Söding. "MMseqs2 enables sensitive protein sequence searching for the analysis of massive data sets." Nature biotechnology, 2017.

[2] Rogers, David, and Mathew Hahn. "Extended-connectivity fingerprints." Journal of chemical information and modeling, 2010.

[3] Bemis, Guy W., and Mark A. Murcko. "The properties of known drugs. 1. Molecular frameworks." Journal of medicinal chemistry, 1996.

AuthorsOpenReview ↗

Response to Reviewer 9gu8 (2)

To Reviewer 9gu8

Following our earlier response, we have now completed the remaining experiments and address the last three points below.


[W4 & Q4] Ablation preserving the feature configuration, and the origin of residue-specificity

We agree the two effects were confounded. The coupling is stated in the caption of Table 4, but stating it does not resolve it. We therefore rerun the ablation on NeuralPLexer with the injected feature dimension held fixed: the ablated component is retained and zero-filled, so the input dimension of WinjW_{inj} and the Stage-2 schedule are unchanged, and the only variable is which component carries information.

φTop-1 25th ↓Top-1 50th ↓Top-1 %<5Å ↑Top-1 %<2Å ↑Top-5 25th ↓Top-5 50th ↓Top-5 %<5Å ↑Top-5 %<2Å ↑
[none]1.653.2165.136.61.553.0975.146.0
[0; e]\mathbf{e}]1.503.1766.036.71.402.7775.945.9
[zc\mathbf{z}_c; 0]1.472.9967.237.91.132.4476.446.0
[zc\mathbf{z}_c; e\mathbf{e}]1.302.6269.540.21.052.1477.347.9

Each component alone gives a modest gain, and the full representation is best on all eight columns: the text component leads in isolation (37.9 against 36.7 on top-1 RMSD<2Å), but neither approaches the 40.2 obtained by combining them. The pattern in the submitted Table 4 therefore does not depend on the coupling, and we will add this table to the revision.

Setting up this experiment also made clear that the construction of ϕ\phi was underspecified, as Q4 notes. We have revised the passage around Equations (5) and (6) to give the pooling, the broadcasting of zc\mathbf{z}_c along the residue axis, and the tensor dimensions explicitly.

The reviewer also asks how the contrastive objective encourages residue-specific interaction learning. The contrastive objective (LInfoNCEL_\text{InfoNCE}) operates at the complex level, so on its own it does not align individual residues with residue-specific descriptions. What makes the representation residue-resolved is the per-residue geometry objective (LgeoL_\text{geo}) it is paired with in Equation (3): the two supply, respectively, which interactions are present and where they occur. A genuinely residue-level contrastive objective, matching pocket residues against the phrases that name them, is a natural extension, which we plan to pursue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.