- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes PocketMon, a cryptic pocket discovery framework that integrates local geometric features with global protein representations. It combines residue level pocket prediction with ligand ranking and ligand induced conformation modeling, demonstrating competitive performance across multiple benchmarks and potential for proteome scale cryptic pocket discovery.
Critical strengths
- Well-motivated architectural design. Integrating an SE(3) equivariant geometric encoder with AlphaFold2 pairwise representations effectively combines local geometric features with global protein context.
- Comprehensive evaluation and ablation studies. The method is evaluated on multiple datasets, including PocketMiner, CryptoBank, and AF2BIND, with extensive ablation studies examining key components.
- Superior performance over baselines. PocketMon consistently outperforms multiple baseline methods across different benchmarks, demonstrating its effectiveness in cryptic pocket prediction.
- Effective cryptic pocket identification in dynamic proteins. PocketMon identifies latent binding sites involving domain rearrangements and loop remodeling without costly conformational ensemble sampling.
Critical weaknesses
- Potential limitation of static geometric representations for cryptic pocket prediction. PocketMon constructs local geometric features and residue interaction graphs from a single input structure, whereas cryptic pocket formation may involve substantial conformational rearrangements that alter inter-residue distances, local neighborhoods, and backbone orientations. Consequently, residue pairs that become spatially proximate in the pocket forming state may not be directly connected in the input graph, and their target state geometry is not explicitly represented by the local geometric encoder.
- Limited methodological novelty. The proposed framework primarily combines established components, including SE(3) equivariant geometric encoding, residue pair representations from AlphaFold2, and GotenNet style interactions. Although their integration is reasonable, the paper does not sufficiently clarify the methodological advances beyond combining existing techniques. In particular, the conceptual formulation of cryptic pocket discovery as local global geometric inference does not, by itself, establish a substantially novel learning mechanism.
- The computational scalability of the global pairwise representation is insufficiently characterized. Incorporating residue pair features provides valuable long range context, but pairwise representations scale quadratically with protein sequence length. Given the proteome scale application, reporting inference time, memory consumption, and performance across proteins of different lengths would help establish the practical efficiency of the framework.
- Limited experimental validation of newly predicted pockets. The proposed workflow relies mainly on computational benchmarks, molecular ranking, and structure prediction. The large scale proteome analysis reports potential pockets and estimates their druggability using computational criteria, but these results do not establish that the predicted pockets are experimentally accessible or capable of binding ligands. Experimental validation or stronger independent evidence for selected novel predictions would substantially strengthen the biological claims.
Questions for authors
- Static Geometry. How does PocketMon account for substantial conformational rearrangements during cryptic pocket formation?
- Methodological Novelty. Beyond integrating established components such as SE(3) equivariant encoding, AlphaFold2 derived residue pair representations, and GotenNet style interactions, what are PocketMon's key methodological advances?
- Scalability. What are the inference time and memory costs for proteins of varying lengths?
5 comments
W3&Q3 (1/2). Given quadratic pair-representation costs, report inference time, memory, and predictive performance across protein lengths.
Thanks for your important feedback on computational scalability. We have added per-protein timing, GPU memory measurements, and length-stratified performance evaluations across all three test sets. We report PocketMon's inference time and memory overhead separately across proteins of varying lengths, while distinguishing the AF2 feature-extraction cost from the PocketMon inference cost using cached features. The protocol and results are provided in Appendix D.3.1, Tables 5–7.
We performed all measurements on a single NVIDIA A100-SXM4-40GB GPU using the original fixed checkpoints, original test splits, batch size 1, and FP32 precision. Each protein was run at its actual length, warmed up 3 times, and timed synchronously 10 times. We first took the median per protein and then aggregated across predefined length bins. The inference time in Table 5 (reproduced below) includes graph construction, feature conversion and transfer to GPU, the geometric backbone, and the fusion readout. Memory is reported as the maximum PyTorch peak allocated memory within each bin, covering the model, inputs, and intermediate tensors.
| Dataset | Residue Length Bin | Number of Timed Proteins | Median Inference Time (ms) | Max Peak Tensor Memory in Bin (GiB) |
|---|---|---|---|---|
| PocketMiner | 1–200 | 12 | 14.35 | 0.49 |
| PocketMiner | 201–400 | 17 | 23.98 | 0.86 |
| PocketMiner | 401–600 | 4 | 33.11 | 1.34 |
| PocketMiner | 601–800 | 2 | 41.85 | 1.63 |
| CryptoBank | 1–200 | 326 | 14.14 | 0.43 |
| CryptoBank | 201–400 | 297 | 21.69 | 1.04 |
| CryptoBank | 401–600 | 112 | 31.73 | 1.52 |
| CryptoBank | 601–800 | 24 | 44.60 | 2.04 |
| CryptoBank | >800 | 11 | 58.06 | 3.16 |
| AF2BIND | 1–200 | 8 | 14.87 | 0.42 |
| AF2BIND | 201–400 | 44 | 23.85 | 1.00 |
| AF2BIND | 401–600 | 16 | 31.14 | 1.23 |
Full-feature timing covers 35 PocketMiner, 770 CryptoBank, and 68 AF2BIND test proteins. Taking the longest full-feature sample in CryptoBank (1,256 residues) as an example, inference takes 77.82 ms with a peak tensor memory of 3.16 GiB. These results support single-GPU per-protein processing across the tested length range. We further measured the extraction cost of residue-pair features separately on three real apo structures of different lengths using the same GPU model (Table 6, reproduced below).
| Protein Length (Residues) | First Prediction & Feature Retrieval Time incl. JIT (s) | Peak Feature Extraction Process Memory (GiB) |
|---|---|---|
| 127 | 17.80 | 1.51 |
| 257 | 17.74 | 1.51 |
| 661 | 34.84 | 4.64 |
The timing includes the initial JIT compilation and excludes model initialization, input preparation, and file saving. Memory uses an on-demand allocation setting, sampling the total memory usage of the extraction process every 20 ms via NVML. These measurements clarify the one-time feature-extraction cost, while the generated features can be saved offline and reused, enabling subsequent PocketMon screening via per-protein loading and millisecond-level prediction.
W3&Q3 (2/2). Given quadratic pair-representation costs, report inference time, memory, and predictive performance across protein lengths.
We also re-evaluated all test proteins grouped by length under the original evaluation protocol. Taking the largest dataset, CryptoBank, as an example, the predictive performance across length bins is reported in Table 7 and reproduced below:
| CryptoBank Residue Length Bin | Number of Proteins | ROC-AUC (%) | PR-AUC (%) | Recovery (%) |
|---|---|---|---|---|
| 1–200 | 326 | 71.30 | 22.36 | 20.06 |
| 201–400 | 297 | 74.09 | 15.97 | 16.98 |
| 401–600 | 112 | 73.73 | 10.39 | 13.40 |
| 601–800 | 24 | 78.02 | 9.65 | 14.36 |
| >800 | 11 | 80.08 | 17.34 | 19.96 |
The longest bin in CryptoBank still achieves 80.08% ROC-AUC, and the three non-empty length bins in AF2BIND achieve ROC-AUCs of 92.51%, 94.16%, and 92.94%. These supplementary results show that PocketMon maintains millisecond-level inference speed and manageable memory requirements across the tested protein length range while preserving effective predictive performance on longer proteins, providing practical support for large-scale per-protein screening.
W4. Computational scores do not establish accessibility or ligand binding. Provide experimental validation or stronger independent evidence for newly predicted pockets.
Thank you for this important suggestion. We agree that computational predictions alone do not establish experimental accessibility or ligand binding. Our evaluation provides complementary evidence through residue-level benchmarks (Table 1), component ablations (Figure 3), site-level Top-K analysis (Appendix D.11.3, Table 13), and an end-to-end evaluation on 20 apo–holo targets (Appendix D.2.1, Table 4). Retrospective experimental structural evidence for a selected unmatched prediction is provided in Appendix D.11.2.
These results support predictive utility and selected structural hypotheses, but do not validate all newly predicted sites. We will therefore frame the proteome analysis as candidate prioritization and clearly distinguish retrospective evidence from prospective validation. We plan to pursue experimental assessment of selected predictions in future work to strengthen their biological interpretation.
Thank you for highlighting this distinction. Predicting pocket-forming residues from an initially closed structure is precisely PocketMon’s objective. The model identifies where a cryptic site may form; it does not explicitly simulate the transition or require open-state contacts to exist in the input graph.
Our hypothesis is that static structures contain predictive cues about pocket-forming propensity, including residue composition, local packing, and backbone orientations. Supervised training learns associations between these features and annotated cryptic residues. The global pair representations additionally provide context beyond the local graph’s spatial cutoff, allowing information about non-neighboring residues to contribute to prediction (
Sections 4.1–4.2). Our ablations and pair-only comparisons support the complementarity of these information sources (Figure 3;Appendix D.6.1,Table 11). Nevertheless, neither branch explicitly reconstructs future contacts, and substantial rearrangements with weak observable cues remain challenging. We will clarify this distinction between predicting cryptic-site propensity and modeling the conformational transition.Thank you for requesting clarification. We would like to clarify that our methodological contribution comprises three complementary advances tailored to cryptic-pocket discovery (
Sections 4.1–4.3).Together, these contributions connect the representation of a closed structure, the localization of latent binding regions, and the generation of open-state structural hypotheses. We will clarify these task-specific advances and distinguish them from the established components on which they build.