- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents a hierarchical method for virtual screening. The newly trained model first identifies the high-probability ligand cluster (ligand code), then uses the oracle function to perform an MCTS across all candidates. The model is trained with ligand-protein surface pairing with augmented negative samples so that the model could be used for both known pocket screening and pocket-unknown screening. Evaluation is highly biased by in silico docking score and some experimental settings might be questionable.
Critical strengths
- PRECISE outperforms both ground truth pocket baselines and sequence-level models.
- Rank according to ligand code - search with oracle functions is an innovative 2-stage framework.
Critical weaknesses
- Most of the method componentss are dependent on previous works. e.g. the ligand code design is depended on CoNCISE. The MCT is using ConPLex embedding
- The method section of the paper needs more polish and structured writing to make it easier to follow the whole framework.
- Unaligned training data might be a problem. None of the baseline methods is trained on the same PLINDER training split. This also raises - the question of ablation studies on different components of the model.
- Analysis on the quantized ligand space should be provided here.
Questions for authors
- How would the method perform if we use ConPLEX embedding to train the ligand-surface scoring framework in the beginning so that we do not need a 2-stage procedure
- How would the method perform compared with other baseline methods on DUD-E benchmark (EF metric)?
2 comments
I really appreciate the author's additional results on EF and de-leaked datasets. However, the results in LIT-PCBA benchmark is quite discouraging. I will maintain my rating regarding limited novelty based on related works.
Response to Reviewer 2ja3
Unaligned training data might be a problem. None of the baseline methods is trained on the same PLINDER training split. This also raises - the question of ablation studies on different components of the model
The new Tables 1 and 2 above in the response to Reviewer GvH2 , show the performance of PRECISE in all the DrugClip related experiments, on the same training/testing/validation datasets as in the original DrugClip paper, including using their exact version of DUD-E where they made sure there was no leakage.
Analysis on the quantized ligand space should be provided here.
Can the reviewer clarify what they are looking for here please? We are not sure we understand what is being asked for.
Q1 How would the method perform if we use ConPLEX embedding to train the ligand-surface scoring framework in the beginning so that we do not need a 2-stage procedure
This is very similar to the PRECISE-CODE versus PRECISE-NOCODE study that we did (see Appendix, section C2, table), We find that the codes can improve the embedding, so the ConPLEX embedding would not perform as well.
Q2 How would the method perform compared with other baseline methods on DUD-E benchmark (EF metric)?
As remarked to both other reviewers, we now have an extensive comparison (see Tables 1 and 2 in the response to Reviewer GvH2) and we are performing better, however, as also remarked we don’t think doing better in EF metric on DUD-E is the main contribution of PRECISE. Rather, the main contribution of PRECISE is that we are the only method that can handle multichain and metalloproteins; plus we can screen all of Enamine in about 30 minutes, whereas competitors will be roughly 50 times slower.