- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces PLI-CLIP, a framework for learning transferable, ligand-conditioned protein representations from interaction descriptions. The authors construct PLI-TD by converting PLIP-derived protein–ligand interaction records into textual descriptions, and jointly train a protein–ligand encoder using text–complex contrastive alignment and per-residue geometry prediction. The resulting features are injected into downstream docking models through a residual adapter, followed by backbone fine-tuning. Experiments on DiffDock, NeuralPLexer, and FlowDock show improvements in docking accuracy and physically valid pose prediction, without requiring interaction descriptions or bound poses at inference.
Critical strengths
A useful approach to transferring interaction supervision into docking representations. The framework uses interaction annotations available from resolved training complexes while avoiding their requirement at inference. Combining ligand conditioning, interaction-text alignment, and residue-level geometry prediction provides a coherent approach to enriching sequence-derived protein features. This addresses a relevant limitation of ligand-agnostic protein representations.
Positive transfer across multiple docking architectures. Improvements are reported for three distinct backbones rather than a single tailored model. Particularly encouraging results include DiffDock’s top-1 success rate increasing from 24.36% to 30.31% with ESMFold protein structures, and FlowDock’s relaxed, accurate-and-PB-valid fraction increasing from 50.4% to 54.1%. These results support the practical usefulness of the representation, while the inclusion of physical-validity metrics strengthens the evaluation beyond pose RMSD alone.
Critical weaknesses
The specific contribution of natural-language supervision remains insufficiently isolated. Comparing different textual renderings does not establish whether language provides an advantage over directly encoding or predicting the same structured PLIP interaction records. Moreover, Table 4 removes both an objective and its corresponding feature component, confounding supervision with the downstream feature configuration. These limitations weaken the attribution of the gains to the proposed language–geometry combination, although they do not negate the utility of the complete framework.
The robustness of the reported improvements needs clearer support. The paper does not report variability across runs or confidence intervals, which matters for interpreting relatively modest gains in stochastic docking models. While excluding evaluation complexes from pretraining is a positive precaution, exact-complex exclusion alone does not characterize generalization across similar proteins and ligands. More explicit reporting of baseline fine-tuning conditions and train–test similarity would strengthen confidence in the comparisons.
Questions for authors
Clarification: Were the baselines subjected to the same additional fine-tuning schedule, EMA, and checkpoint-selection procedure as the PLI-CLIP-enhanced models? Please specify how the training and selection budgets were matched.
Evidence request: Do you have a comparison against directly using the same structured PLIP records as supervision? An objective ablation that preserves the downstream feature dimensions would also help distinguish the effects of supervision from changes in the injected representation.
Clarification: Can you provide uncertainty estimates for the main improvements, and characterize protein-sequence and ligand-scaffold similarity between training and evaluation sets? These would help assess whether the gains are stable and how broadly they generalize.
Related-work suggestion, separate from the assessment-changing questions: The discussion of structure-aware protein pretraining could be broadened to include Pre-Training Protein Bi-level Representation Through Span Mask Strategy On 3D Protein Chains (ICML 2024), which studies joint residue- and atom-level representations through span masking. Discussing this work would help position PLI-CLIP relative to complementary approaches for enriching protein representations with structural information, while clarifying its distinct emphasis on ligand-conditioned interaction supervision
5 comments
Response to Reviewer NbTu (2)
To Reviewer NbTu
We have now completed the remaining experiments and address the last two points below. Thank you for your patience.
[Q2 & W1] Structured-record supervision, and an ablation at fixed feature dimension
- Structured PLIP records as supervision.
We trained a variant on DiffDock that encodes the same PLIP records directly as interaction vectors, bypassing the text rendering and the text encoder, and aligns the protein–ligand representation to those vectors under the identical contrastive objective. The target is a 140-dimensional vector counting interactions for each of the twenty residue types paired with each of seven categories, which refine the paper’s six PLIP types by splitting hydrogen bonds on the donor side. It replaces the sentence embedding as the InfoNCE target, retaining the interaction content the captions verbalise. The encoder, the Stage-2 adapter, and the fine-tuning schedule are unchanged, so the only variable is whether the record passes through language.
| Model | CLIP Training | Holo Top-1 %<2Å ↑ | Holo Top-1 Med ↓ | Holo Top-5 %<2Å ↑ | Holo Top-5 Med ↓ | Apo Top-1 %<2Å ↑ | Apo Top-1 Med ↓ | Apo Top-5 %<2Å ↑ | Apo Top-5 Med ↓ |
|---|---|---|---|---|---|---|---|---|---|
| DiffDock | – | 39.89 | 2.93 | 47.01 | 2.16 | 24.36 | 6.01 | 35.41 | 3.66 |
| DiffDock+PLI-CLIP | vector encoding | 40.17 | 2.85 | 48.15 | 2.07 | 28.90 | 5.01 | 37.96 | 3.14 |
| DiffDock+PLI-CLIP | PLM | 41.88 | 2.69 | 49.57 | 2.02 | 30.31 | 4.72 | 38.81 | 3.22 |
Encoding the records directly already improves over the backbone, most visibly on ESMFold structures (24.36 to 28.90), so the interaction supervision transfers and is not specific to language. Language supervision is better still on seven of eight columns, reaching 41.88 on holo and 30.31 on apo top-1; the exception is the apo top-5 median (3.14 against 3.22). We therefore do not claim language as the sole source of the gain, only that interaction records seen at training time transfer into a representation needing no annotation at inference, and that text is the more effective vehicle.
- Ablation at fixed feature dimension.
We agree that Table 4 confounded the supervision with the injected feature, and we have rerun it on NeuralPLexer with the feature dimension held fixed: the ablated component is retained and zero-filled, so the input dimension of and the Stage-2 schedule are unchanged, and the only variable is which component carries information.
| φ | Top-1 25th ↓ | Top-1 50th ↓ | Top-1 %<5Å ↑ | Top-1 %<2Å ↑ | Top-5 25th ↓ | Top-5 50th ↓ | Top-5 %<5Å ↑ | Top-5 %<2Å ↑ |
|---|---|---|---|---|---|---|---|---|
| [none] | 1.65 | 3.21 | 65.1 | 36.6 | 1.55 | 3.09 | 75.1 | 46.0 |
| [0; ] | 1.50 | 3.17 | 66.0 | 36.7 | 1.40 | 2.77 | 75.9 | 45.9 |
| [; 0] | 1.47 | 2.99 | 67.2 | 37.9 | 1.13 | 2.44 | 76.4 | 46.0 |
| [; ] | 1.30 | 2.62 | 69.5 | 40.2 | 1.05 | 2.14 | 77.3 | 47.9 |
Each component alone gives a modest gain and the full representation is best on all eight columns: the text component leads in isolation (37.9 against 36.7 on top-1 RMSD<2Å), but neither approaches the 40.2 obtained by combining them. The pattern reported in the submitted Table 4 therefore does not depend on the coupling.
Both tables will be added to the revision.
[Q3] Uncertainty estimates and train–test similarity
- Variability across runs.
To address the reviewer NbTu’s concern, we repeated Stage-2 training with three random seeds under an otherwise identical configuration on all three backbones (DiffDock, NeuralPLexer, FlowDock).
| DiffDock | Runs | Holo Top-1 %<2Å ↑ | Holo Top-1 Med ↓ | Holo Top-5 %<2Å ↑ | Holo Top-5 Med ↓ | Apo Top-1 %<2Å ↑ | Apo Top-1 Med ↓ | Apo Top-5 %<2Å ↑ | Apo Top-5 Med ↓ |
|---|---|---|---|---|---|---|---|---|---|
| DiffDock | – | 39.89 | 2.93 | 47.01 | 2.16 | 24.36 | 6.01 | 35.41 | 3.66 |
| DiffDock + PLI-CLIP | 1 | 41.88 | 2.69 | 49.57 | 2.02 | 30.31 | 4.72 | 38.81 | 3.22 |
| DiffDock + PLI-CLIP | 3 | 42.07 ± 0.11 | 2.77 ± 0.01 | 48.91 ± 1.32 | 2.04 ± 0.001 | 28.80 ± 1.79 | 4.75 ± 0.007 | 39.47 ± 0.83 | 3.06 ± 0.02 |
| NeuralPLexer | Runs | Top-1 25th ↓ | Top-1 50th ↓ | Top-1 %<5Å ↑ | Top-1 %<2Å ↑ | Top-5 25th ↓ | Top-5 50th ↓ | Top-5 %<5Å ↑ | Top-5 %<2Å ↑ |
|---|---|---|---|---|---|---|---|---|---|
| NeuralPLexer | – | 1.65 | 3.21 | 65.1 | 36.6 | 1.55 | 3.09 | 75.1 | 46.0 |
| NeuralPLexer + PLI-CLIP | 1 | 1.30 | 2.62 | 69.5 | 40.2 | 1.05 | 2.14 | 77.3 | 47.9 |
| NeuralPLexer + PLI-CLIP | 3 | 1.36 ± 0.07 | 2.61 ± 0.04 | 69.2 ± 0.3 | 40.5 ± 0.3 | 1.09 ± 0.04 | 2.19 ± 0.15 | 76.9 ± 0.4 | 47.2 ± 0.5 |
| FlowDock | Runs | Holo RMSD<2Å ↑ | Holo RMSD<2Å & PB-Valid ↑ | Apo RMSD<2Å ↑ | Apo RMSD<2Å & PB-Valid ↑ |
|---|---|---|---|---|---|
| FlowDock | – | 71.4 | 34.0 | 71.0 | 50.4 |
| FlowDock + PLI-CLIP | 1 | 72.2 | 38.4 | 72.2 | 54.1 |
| FlowDock + PLI-CLIP | 3 | 72.6 ± 0.51 | 38.2 ± 0.39 | 71.9 ± 0.37 | 53.0 ± 1.33 |
Across three seeds the improvements hold on every backbone. NeuralPLexer is the most stable, with standard deviations of 0.3–0.5 points on the below-threshold metrics and a mean of 40.5 against the backbone’s 36.6. FlowDock varies by 0.4–1.3 points and stays above the backbone throughout. DiffDock is the widest, with apo top-1 at 28.80 ± 1.79 against the single run’s 30.31, but the backbone reaches only 24.36 on the same evaluation, so the margin exceeds the spread. We will report mean and standard deviation in the revision.
- Train–test similarity.
For every test complex we measured the maximum protein sequence identity with MMseqs2 [1] and the maximum ligand Tanimoto similarity over Morgan fingerprints [2] against PLI-TD, and separately against each backbone's own training set, together with Bemis–Murcko scaffold matches [3].
| Test set vs PLI-TD | Max seq id > 30% | Max seq id = 100% | Max Tanimoto > 0.7 | Scaffold match |
|---|---|---|---|---|
| PDBBind2020 (NeuralPLexer) | 74.4% | 43.0% | 29.2% | 36.1% |
| PoseBusters (FlowDock) | 54.9% | 16.6% | 27.9% | 43.2% |
| Test set vs training set | Max seq id > 30% | Max seq id = 100% | Max Tanimoto > 0.7 | Scaffold match |
|---|---|---|---|---|
| PDBBind2020 (NeuralPLexer) | 84.8% | 39.9% | 20.7% | 33.1% |
| PoseBusters (FlowDock) | 59.1% | 21.1% | 34.4% | 47.4% |
Residual similarity is considerable. On PDBBind2020, 43.0% of test complexes share an identical protein sequence with some PLI-TD entry under a different PDB code; the corresponding figure against the backbone's own training data is 39.9%, so PLI-TD adds little overlap beyond what the benchmark already carries. PoseBusters, drawn from later depositions, is markedly cleaner at 16.6%. We report these figures as they are. The comparison locates where the overlap comes from, but it does not tell us what that means for generalisation.
To address the latter, we removed the overlapping complexes and re-evaluated NeuralPLexer and FlowDock on the remaining disjoint subsets, filtering once by sequence identity and once by ligand similarity. The models are unchanged and only the evaluation set differs.
Response to Reviewer NbTu
To Reviewer NbTu
We thank the reviewer for the careful reading and thoughtful feedback. We are currently running experiments to address these concerns, and we report those already complete so that the discussion can begin. We will respond to the remaining points (Question 2 and 3) within this week.
[Q1&W2] Matched fine-tuning schedule, EMA and checkpoint selection.
In the submitted version, the baselines were evaluated from the officially released weights of each backbone, without the additional Stage-2 fine-tuning or EMA applied to the PLI-CLIP models. We agree that this left the comparison unmatched, and we have now added the matched control for all three backbones. Each baseline is fine-tuned for the same 10 epochs at the same learning rate (1e-4) on the backbone's own training data, evaluated with the same EMA of the fine-tuned weights, and selected by the same rule, so the only difference is the presence of the pretrained interaction representation. The Fine-tuning column marks whether Stage-2 fine-tuning was applied.
As the three results above show, fine-tuning on its own does not account for the improvements. Performance is essentially unchanged across all three backbones, and on DiffDock with ESMFold structures top-1 accuracy in fact drops from 24.36 to 23.51. NeuralPLexer moves only from 36.6 to 36.7, and FlowDock from 34.0 to 34.2 on the accurate-and-PB-valid fraction. With PLI-CLIP, by contrast, all three backbones improve consistently under the identical schedule.
[Q4] Related work: span-mask bi-level protein pretraining (ICML 2024).
We have added a discussion of this work to Section 2, marked in blue. Its supervision is intrinsic to the protein chain, which makes it complementary to our approach, where the supervision is conditioned on a specific bound ligand and derived from observed interaction records, so it was indeed worth including. We thank the reviewer for the suggestion.