- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces zPocket, a method that predicts protein–ligand binding pockets using lightweight prediction heads trained on frozen representations from cofolding models. The authors also develop PocketWatch, a benchmark designed to evaluate generalization to novel binding pockets. On 1275 test proteins covering 1562 pockets, zPocket achieves a per-target AUPRC of 0.757, outperforming the several retrained baselines, such as 0.626 for VN-EGNN, and 0.503 for P2Rank. Combining zPocket with existing pocket detectors further improves pocket localization, reaching a top-(N+2) recall of 0.666. The authors also evaluate the method on 43 fragment-screening pockets and apply it to approximately 20000 human proteins, identifying 155 potentially novel pockets after filtering.
Critical strengths
-
I think the methodology proposed by the authors is rational. Existing Cofolding models, trained via large-scale pretraining, may have already learned extensive knowledge of protein-small molecule interactions. Directly leveraging the internal representations of these models to predict binding pockets represents a reasonable and practically valuable research direction. Notably, the model can produce residue-level Pocket Probability without requiring explicit ligand input. Furthermore, the empirical results are good. After retraining VN-EGNN and UniSite on the same dataset, zPocket still exhibits clear advantages. Consistent observations across different frozen backbones also strengthen the credibility of the conclusions.
-
I think the PocketWatch benchmark is potentially a more valuable contribution than the model design itself. Rather than simply improving prediction performance, the authors address an important question in the era of cofolding models: whether a model can truly identify novel binding pockets or is simply recognizing patterns already seen during pretraining. PocketWatch uses temporal splitting and stratifies the test set by sequence similarity, structural similarity, and crypticity, providing a clearer picture of model performance on pockets with different levels of difficulty. The authors also retrain several baselines and explore applications in fragment screening and proteome-scale pocket discovery. Overall, I appreciate the comprehensive experimental design and the effort to evaluate the model's generalization ability.
Critical weaknesses
-
This is my main concern with the paper. Although the authors emphasize novel pocket prediction, 71.6% of the test pockets have high structural similarity to previously known binding sites. Performance drops substantially on pockets with little structural precedent, where the improvements over baselines become smaller and some confidence intervals overlap. Therefore, the overall AUPRC of 0.757 may not fully reflect the model's ability to discover genuinely novel pockets. The evaluation of cryptic pockets is also limited by the small sample size: only 40 of the 1,562 test pockets are labeled as cryptic, and just 20 of these lack holo-like conformations in the predicted structures. I believe more convincing evidence on genuinely novel and challenging cryptic pockets is needed to support the paper's main claims.
-
While the overall research direction is promising, I feel that the proposed methodology offers limited technical novelty. Leveraging pair representations from pretrained structure prediction models and training lightweight prediction heads using existing architectures is a relatively straightforward approach. Similar ideas have also been explored in prior work, such as AF2Bind. Although zPocket achieves impressive results, most of the compared baselines follow different technical approaches, and comparisons with closely related methods that also leverage representations from pretrained structure prediction models are limited. The current ablations demonstrate the benefits of pair representations and different head architectures, but do not fully clarify where the improvements come from. I believe the authors should better highlight the unique technical contributions of zPocket and provide stronger comparisons to distinguish methodological improvements from the advantages of using powerful pretrained cofolding models.
Questions for authors
-
Since leveraging pretrained pair representations for downstream prediction tasks is not entirely new, could the authors further clarify the main methodological innovations of zPocket? In particular, how much of the performance improvement comes from the pretrained cofolding representations versus the prediction head architecture? Additional ablations using alternative representations, such as predicted structures or distogram features, could help clarify this distinction.
-
The authors compare against UniSite-1D but do not include UniSite-3D, which incorporates structural information and supports both residue-level and pocket-level predictions. In addition, AF2Bind, which also leverages pair representations from pretrained structure prediction models for binding-site prediction, is discussed in the paper but not included in the experiments. Could the authors clarify why these methods were excluded and, if feasible, provide additional comparisons? I believe these comparisons would help better establish the performance advantages and technical contributions of zPocket.
2 comments
We’ve edited our reply above to lay out more clearly how this work extends prior work, to clarify our point about cryptic pockets, and to add how we revised the introduction to surface the point about structurally novel pockets sooner. With the discussion period closing tomorrow, we hope you will review our changes, and if our added experiments address your concerns, that you’ll consider raising your score.
Reply to Reviewer vwoU
Thank you for the detailed suggestions. The requested ablations and baselines are now in the paper. We also give the headline numbers here and cite the common response for the full tables.
W1: Novel and cryptic pockets. We agree our “novel” pocket set is dominated by structurally similar pockets, and our goal in creating the benchmark was to make performance as a function of structural similarity visible to the field. Excluding the high-coverage bin leaves 443 pockets, on which zPocket reaches 0.445 per-pocket AUPRC versus 0.337 for retrained UniSite-3D, the strongest baseline: a paired gain of +0.109 [95% CI: 0.091, 0.126]. In the hardest bin (0–20%; 133 pockets), zPocket leads the strongest baseline, DeepPocket, by +0.084 [0.036, 0.135], Holm-adjusted p = 1.5e-4. Common response §1 reports these paired tests alongside the marginal intervals. We also revised our introduction to surface this point, as obscuring it was never our intention.
We agree that the 40-pocket cryptic subset is too small for a strong generalization claim. Appendix B.9 further shows that much of the apparent crypticity effect is associated with predicted-fold accuracy. We therefore treat these results as preliminary and report the sample sizes explicitly. We also note that the existing works from the industrial labs that report performance on cryptic pockets do not report the size of this evaluation set, and we think one of our contributions is revealing the small number of pockets available for such evaluations, adding more context to the numbers reported from other labs.
W2 / Q1: Technical contribution and ablations. We know similar work using paired representations has been done, but we expand on it in several ways: we create a benchmark suitable for the new generation of models and show their gains in performance, and we show how these per-residue scores can be combined with discrete pocket-finding methods to better prioritize pockets, which is practically useful for the field beyond comparing per-residue metrics. We also added your requested analyses to better understand the contribution of the representations versus the prediction head. The requested distogram control reaches 0.701 per-target AUPRC versus 0.745 for pair features under the same pairformer2 architecture, a paired gain of +0.044 [95% CI: 0.037, 0.052]. A two-block sequence-transformer head reaches 0.633 on ESMC-6B embeddings despite roughly 2.5 times the parameters of the pair head. Even with mixing, sequence-based representations cannot match the structural pair representations, suggesting the pair features truly contain useful binding signal. Common response §3 gives the full ablations.
Q2: UniSite-3D and AF2Bind. Both are now included (common response §2), reaching 0.544 and 0.456 per-target AUPRC with published weights, versus 0.745 for zPocket. We also retrained UniSite-3D on PocketWatch, and it reaches 0.648 [0.631, 0.664], a gain of +0.103 [+0.094, +0.113] over its own published weights, which makes it the strongest baseline in Table 1. zPocket still leads it by 0.098 [0.089, 0.106]. Adding UniSite-3D was the right thing to do, and we are grateful for the suggestion. For AF2Bind, we also added a comparison on AF2Bind's own benchmark: 0.738 Recovery against 0.662 for AF2Bind (also described in common response §2).
The distogram control and the two missing baselines were the right things to ask for, and the paper is stronger for them. We hope these address your concerns and that you will consider raising your score. We would also be glad to discuss any other details further.