- Rating
- 3
- Confidence
- 2
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes CoPocket, a pocket identification framework using protein sequence and MSA with a co-folding pretrained representation, without the need for the structure. The authors also introduce ChronoPocket, a temporal benchmark that distinguishes newly observed pockets from repeated observations of known binding sites. Overall, the paper addresses a meaningful problem and demonstrates promising experimental results to support the main claim.
Critical strengths
- The problem is meaningful, predicting pockets without requiring receptor structures or known ligands is valuable for early-stage drug discovery.
- This work builds a large-scale dataset ChronoPocket for the pocket identification task with temporal evaluation and protein-centered labels.
- CoPocket outperforms the ESMC-based baseline on ChronoPocket and achieves better performance than UniSite on the evaluated external subset. The evaluation covers both pocket-level and residue-level metrics.
- The motivation in this paper is relatively clear, and the figure design is elaborate.
Critical weaknesses
-
The description and the contribution of the co-folding pretrained model are still somewhat not clear. I understand that the co-folding model, such as IntelliFold V2-Flash, is pretrained on protein-ligand complexes and also the protein-only data, therefore, it may contain contact information for the protein input. The paper emphasizes the use of co-folding representations, but CoPocket only extracts protein-only representations from IntelliFold V2-Flash without ligand. And the reported improvements demonstrate the effectiveness of the overall CoPocket system but do not establish how much of the gain is attributable specifically to the pretrained co-folding representations. It remains unclear whether the effectiveness of these representations is specifically related to protein–ligand interaction knowledge acquired during co-folding pretraining, or primarily to general protein structural information. Although the paper compares the performance between IntelliFold V2-Flash and ESMC, a pure PLM, the contribution is not explicitly and directly evaluated.
-
The proposed framework demonstrates promising results using pretrained representations from IntelliFold V2-Flash. However, it remains unclear whether the observed effectiveness generalizes to other co-folding backbones. Evaluating CoPocket with an additional pretrained co-folding model, such as Boltz-1 or Protenix, would further strengthen the core claim of this paper.
-
The contributions of several proposed components, such as pairformer, are not independently evaluated. Moreover, it seems that removing the pair-derived attention bias does not consistently reduce performance. This diminishes the contribution of the method, considering that the significant information mainly comes from the use of a pretrained co-folding model.
-
Although recovering later-observed pockets is an important contribution, the discovery evaluation does not include comparisons with existing methods. Without such baselines, it is difficult to assess the significance of CoPocket especially for discovering additional pockets.
Questions for authors
-
The proposed framework is evaluated using only IntelliFold V2-Flash as its co-folding backbone. Testing additional co-folding models would help assess whether the approach generalizes across different co-folding pretrained representations.
-
Some experimental results or analysis to clarify other weaknesses would be good, if possible.
7 comments
W2 / Q1. Could other co-folding backbones such as Boltz-1 or Protenix be tested to assess cross-backbone generality?
Under the same feature dimensions, 384-residue crop schedule, and 19,840-step training budget, we evaluated frozen Boltz-1 and Protenix v0.5.0 (Table R2; corresponding to Section 5.3, lines 410–455, Table 4 and Appendix F.3, Table S11).
Table R2. Cross-backbone replacements under an identical CoPocket decoder and training budget (ChronoPocket full test, homology-filtered subset n = 1,546, and ChronoPocket-D). Nat. AP denotes native per-residue macro AUPRC; bold marks the best value per column.
| Frozen Backbone | Pretraining & Cutoff | Nat. AP / MCC | Filtered () | Disc. Top-1 (New / Nov.) | Disc. Top-5 New [95% CI] | |||
|---|---|---|---|---|---|---|---|---|
| ESM C 600M (ref.) | Sequence MLM | 0.414 | 0.281 | 0.391 | 0.620 / 0.502 | 0.188 | 5.9 / 5.6 | 9.1 [6.6, 12.0] |
| AlphaFold2 monomer | Protein folding (Pre-2021-09-30) | 0.548 | 0.411 | 0.471 | 0.684 / 0.533 | 0.367 | 5.9 / 5.1 | 22.6 [18.9, 26.8] |
| Boltz-1 | Co-folding (2021-09-30) | 0.515 | 0.397 | 0.454 | 0.688 / 0.562 | 0.318 | 8.0 / 6.9 | 18.8 [15.4, 22.5] |
| IntelliFold V2-Flash | Co-folding (cutoff undocumented in public docs) | 0.547 | 0.435 | 0.489 | 0.704 / 0.571 | 0.362 | 7.0 / 6.4 | 19.7 [16.2, 23.6] |
Replacing IntelliFold V2-Flash with frozen Boltz-1 (Section 5.3, lines 426–431, Table 4) substantially outperforms all 20 external baselines in Table 2 (strongest external baseline ) and the matched ESM C 600M control (0.281) across every metric, achieves the highest ChronoPocket-D Top-1 discovery recall (8.0% / 6.9%), and preserves 18.8% Top-5 recall (18.5% / 19.4% on the homology-filtered subset; Appendix F.3, Table S11 Panel B). This confirms that CoPocket transfers across co-folding trunks rather than depending on IntelliFold V2-Flash alone.
For Protenix v0.5.0, training under the identical unadapted protocol yielded validation , so we excluded it from the unadapted comparison table (Section 5.3, lines 421–425). Specifically, Protenix's raw single-residue state emitted outside its diffusion conditioner lacks trunk-specific normalization, lies at an extreme numerical scale, and exhibits near-collinear token directions, saturating downstream layers of a generic decoder; full output-scale diagnostics are documented in Appendix F.3 (lines 1467–1477).
W3 (+ Q2). Individual modules such as Pairformer lack standalone ablations, and removing pair-derived attention bias did not consistently degrade performance; does this suggest the downstream architecture contributes little?
Under matched initialization and training steps (19,840 steps), we completed full downstream ablations covering Pairformer refinement, the mask-quality head, decoder pair bias, and fixed-window inference controls (Table R3).
Table R3. Controlled downstream ablations and fixed-window inference comparisons at matched training steps (Table 3; extended metrics and threshold sweeps in Tables S8–S10). Bold marks the best value per column.
| Downstream Variant / Inference Control | Filtered () | Nat. AUPRC | Disc.@5 New / Nov.-low (%) | |||
|---|---|---|---|---|---|---|
| CoPocket (matched control) | 0.547 | 0.435 | 0.362 | 0.727 | 0.704 | 19.7 / 18.1 |
| w/o pair-derived attention bias (updated simplified) | 0.553 | 0.439 | 0.370 | 0.736 | 0.705 | 21.1 / 19.9 |
| w/o Pairformer refinement | 0.277 | 0.101 | 0.091 | 0.628 | 0.554 | 4.7 / 4.1 |
| w/o final quality head | 0.518 | 0.394 | 0.333 | 0.723 | 0.699 | 19.9 / 18.4 |
| Objectness-only ranking (fixed window) | 0.526 | 0.403 | — | 0.729 | 0.704 | 19.5 / 18.1 |
| Binary-union window fusion (fixed window) | 0.547 | 0.391 | — | 0.722 | 0.558 | 17.1 / 15.3 |
Table R3 establishes the necessity of the downstream components and clarifies how pairwise information flows: (1) Removing only the two trainable Pairformer blocks while retaining and the decoder bias causes to drop from 0.435 to 0.101 (Nat. AUPRC to 0.554, discovery recall to 4.7%; Section 5.2, lines 371–378, Table 3, line 394; Appendix F.1, Table S8). Together with Table R1, both " without Pairformer" (0.101) and "Pairformer without " (0.321) trail the joint design (0.435), demonstrating that frozen co-folding pair features must be transformed by trainable Pairformer refinement. (2) Because those Pairformer blocks already fold pairwise couplings from into the residue state , the partner-averaged additive decoder bias adds no further gain; we report the bias-free variant achieving 0.439 (21.1% discovery) and formally adopt it as our simplified updated decoder (Section 5.2, lines 375–383, Table 3, line 396; Appendix F.1, line 1403). (3) The quality head calibrates cross-protein ranking (0.394 without it), while confidence-weighted probability fusion outperforms binary union on strict boundary precision (0.435 vs. 0.391) and unified residue AUPRC (0.704 vs. 0.558; Section 5.2, lines 405–409; Appendix F.2, Tables S9–S10).
Dear Reviewer 5o3y, thank you very much for your time and insightful comments. In the revised manuscript, all updated and newly added content is highlighted in blue.
Below, we address each of your weaknesses and questions point by point.
W1. When extracting representations from ligand-free proteins alone, to what extent do the gains come specifically from co-folding pretraining rather than generic protein structural information or downstream modeling?
We tested both questions under an identical decoder architecture, 384-residue cropping schedule, MSA inputs, and training budget (5 epochs / 19,840 steps) via single-factor feature ablations and frozen-backbone replacements (Table R1 and Table R2).
Table R1. Single-factor representation and input ablations under matched downstream architecture and controlled training (ChronoPocket test
n = 2,890, homology-filtered subsetn = 1,546, and ChronoPocket-Dn = 308).Nat. AUPRCdenotes full-sequence per-residue macro AUPRC, andDisc.@5denotes ChronoPocket-D Top-5 recall (%); bold marks the best value per column.First, as shown in Table R1 (Section 5.2, lines 356–370, Table 3; Appendix F.1, Table S8), keeping the MSA and all downstream modules fixed while zeroing out only the frozen co-folding pair representation Z(0) reduces AP0.5 from
0.435to0.321(0.212on the homology-filtered subset) and drops ChronoPocket-D recall to10.6%. Removing the Pairformer layers and adapting residue states S(0) with a depth-matched Transformer encoder further lowers AP0.5 to0.257—comparable to the matched sequence-only language modelESM C 600M(0.281)—while withholding MSAs collapses performance to ≤0.081. This confirms that even without ligand inputs, MSA-conditioned pretrained pairwise features Z(0) supply the primary geometric prior (the adaptation of ligand-free co-folding states is detailed in Section 4.2, lines 202–208). Second, because pretraining data sources cannot be disentangled inside a single checkpoint, we replaced the frozen trunk—under identical (384,128) feature dimensions—with template-free AlphaFold2 monomer, which was pretrained exclusively on protein structures (Table R2, top two non-reference rows; Section 5.3, lines 410–455, Table 4; Appendix F.3, Table S11). AlphaFold2 monomer outlines broad surface cavities effectively (coarse overlap AP0.3=0.548 vs.0.547, maintaining high Top-5 coverage in part by emitting more candidate masks), but falls short on strict pocket boundary precision (AP0.5:0.411vs.0.435), pocket center localization (DCCn:0.471vs.0.489), full-sequence residue selectivity (Nat. AP0.684vs.0.704;MCC0.533vs.0.571), and ChronoPocket-D Top-1 prioritization (5.9%, matching sequence-only ESM C and trailing both co-folding trunks at7.0%–8.0%). Thus, generic monomer folding suffices to locate broad surface clefts, whereas small-molecule co-folding pretraining further sharpens pocket contact boundaries and Top-1 site ranking.