- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies the problem of within-target ligand ranking in structure-based virtual screening, addressing pose uncertainty, target-dependent metric reliability, and heterogeneous supervision density across protein targets. To tackle these challenges, the paper propose PoT-Rank, a target-local learning-to-rank framework that integrates multiple protein–ligand poses and heterogeneous scoring metrics through hierarchical feature fusion, reliability calibration, and hard-decoy curriculum learning, while employing a separate pairwise comparator for sparsely supervised targets. To evaluate its effectiveness, the authors conduct experiments using protein-level MMseqs2 splits at 40% sequence identity, comparing against classical docking scores, deep affinity predictors, graph-based models, pose-aware methods, and tabular baselines, alongside ablation studies, leakage audits, and robustness analyses. The proposed framework achieves within-target pairwise accuracies of 0.6602 on three dense test targets and 0.6897 on 79 sparse test targets, outperforming the evaluated domain-specific baselines in the dense setting and matching XGBoost's pairwise accuracy in the sparse setting.
Critical strengths
Instead of simply performing structure-based virtual screening using various information, such as structural confidence (the reliability of predicted complex structures), docking affinity (energy-based scores of binding poses), interface geometry (protein–ligand contacts and surface area), interaction fingerprints (hydrogen bonds, hydrophobic interactions, and π-stacking), and ligand descriptors (physicochemical properties of compounds), this study attempts to learn which features, poses, and metrics are more useful for ranking. I believe this approach addresses an important direction for future research in virtual screening.
Critical weaknesses
This approach seems to be a form of feature selection and metric selection, and there are insufficient ablation studies or further analyses to demonstrate which types of information have higher reliability. Also, the study seems to address a relatively small search-space problem by integrating 13 existing structure-based features and heterogeneous metrics to identify which information is more useful for target-specific screening. To address this problem more concretely, it would be helpful to conduct broader experiments covering not only SBVS but also LBVS, 3D-based, and diffusion-based methods, and to demonstrate which metrics and feature information are actually important. The current experimental results seem insufficient to fully support these claims.
Also, as has been widely demonstrated in research on deep learning-based VS, various approaches have been proposed to enhance virtual screening performance through systematically constructed representations. As such, without presenting comparison results against SOTA models such as DrugCLIP[1] or PUResNET[2] etc.., it is difficult to ascertain whether the proposed method has achieved a significant improvement in ranking performance compared to these approaches, and if so, how. Including such comparisons would make the experimental results more convincing.
[References]
[1] Gao, Bowen, et al. "Drugclip: Contrastive protein-molecule representation learning for virtual screening." Advances in Neural Information Processing Systems 36 (2023): 44595-44614.
[2] Jeevan, Kandel, et al. "PUResNetV2. 0: a deep learning model leveraging sparse representation for improved ligand binding site prediction." Journal of Cheminformatics 16.1 (2024): 66.
Questions for authors
-
Is the manifest-level regime assignment defined by the authors, or are there previous studies that have divided datasets into these regimes in a similar way? How were the regime hints specified in the manifests determined?
-
If these concepts were introduced specifically for the authors' experimental setup, I think additional experiments under various settings would be necessary. Since the reported results are based on datasets constructed by the authors rather than established benchmark datasets, it is difficult to fully assess the reliability of the experimental results. Based on the dataset statistics, it is also unclear how challenging these tasks actually are(There are insufficient data points).
-
The number of data points is quite small. Have the authors considered data augmentation using AlphaFold-based simulations? There would likely be a gap between real experimental data and simulated information. How do the authors intend to address this gap?
-
For the sparse regime, I think comparisons with methods that perform well under low-label settings, such as SubDyve[1] or network propagation-based approaches, would be useful.
-
Recent studies have shown that subgraph information can help predict protein–ligand binding affinity, and that certain subgraphs can capture pharmacologically meaningful characteristics or contribute to docking affinity prediction. What advantages does the proposed method offer compared to these approaches?
-
SBVS can be computationally expensive, but one of its advantages is that it can provide accurate predictions. However, the number of compounds that need to be screened continues to increase. Therefore, I am also curious whether the proposed method offers any advantages in computational speed or memory efficiency compared to computationally efficient approaches such as DrugHash[2] or other models.
-
It would be beneficial to conduct cold-split experiments on a large-scale set of protein–ligand pairs and analyze the results in comparison with a wider range of existing VS methods. Such experiments would help to demonstrate the generalizability and practical advantages of the proposed framework.
-
There are quite a few model variants in this study, making it difficult to identify which components actually contribute to the performance improvements. Although the paper provides several ablation studies, it is difficult to regard these results as providing sufficient support for the authors’ main claims, particularly regarding the effectiveness of the hierarchical scorer and reliability calibration. It is unclear whether the reported improvements are directly related to these core components or mainly come from the additional policy priors, memory-based calibration, and other variants. A clearer analysis of how each component contributes to the final performance would help to support the paper’s key methodological claims.
[1] Yi, Jungseob, et al. "SubDyve: Subgraph-driven dynamic propagation for virtual screening enhancement controlling false positive." International Conference on Learning Representations. Vol. 2026. 2026.
[2] Han, Jin, Yun Hong, and Wu-Jun Li. "Drughash: Hashing based contrastive learning for virtual screening." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 16. 2025.