- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes ReqVS, a reliability-aware, sequence-based virtual screening framework that integrates heterogeneous drug–target interaction (DTI) and drug–target affinity (DTA) predictors. It combines ensemble scores with a penalty based on averaged per-model binary predictive entropy. The authors report improved early enrichment on DUD-E and DEKOIS2.0 and present a prospective HCAR1 screening campaign involving approximately ten million compounds, followed by expert-guided filtering and cell-based evaluation of cpd60.
The problem is relevant, and the prospective component is potentially valuable. However, I have substantial concerns about the novelty and interpretation of the entropy estimator, the incremental benefit over ensemble scoring, and the pharmacological interpretation of the HCAR1 results.
Critical strengths
- A target-sequence-based approach can be useful when experimentally suitable receptor structures or functional states are unavailable.
- Attempting laboratory validation after a large-scale virtual screen is an important strength relative to benchmark-only studies.
Critical weaknesses
-
Ensemble consensus, predictive entropy, and uncertainty-aware screening have substantial precedents, including SCORCH (McGibbon et al., 2023), SCORCH2 (Chen et al., 2025), and Moshawih et al. (2024). These methods differ from ReqVS, particularly in their structural inputs, but the authors should explain the specific algorithmic novelty beyond combining existing sequence-based predictors with an entropy penalty. Comparisons against straightforward consensus-ranking and uncertainty-penalized baselines are needed.
-
Equations (7)–(8) average the individual models' predictive entropies, , rather than measuring compound-specific disagreement between models. For example, predictions of 0.99 and 0.01 produce low average entropy despite strong disagreement. Consensus weighting considers model agreement at the weighting stage but does not resolve this distinction. Please compare the proposed estimator with ensemble predictive entropy, prediction variance, and mutual-information/Jensen–Shannon disagreement, and substantiate its proposed connection to uncertainty from unobserved binding structures.
-
The incremental benefit of entropy regularization is modest. On DUD-E (Table 3), adding entropy changes mean EF0.1% from 41.64 to 42.65 and BEDROC from 0.454 to 0.459, while AUROC slightly decreases from 0.821 to 0.820. This suggests that ensembling contributes most of the gain. Please provide paired target-level significance tests or confidence intervals, sensitivity analysis for , and evidence that entropy meaningfully improves the top-ranked candidate list.
-
Appendix B.2 uses library-dependent min–max normalization to convert DTA affinity scores into values treated as binding probabilities for Shannon entropy, although such scores are not necessarily calibrated probabilities. The consensus-derived weights may also favor models with shared errors. Please examine probability calibration, normalization and weighting alternatives, and the effect of synthetic-negative augmentation. Given known biases in DUD-E (Chen et al., 2019), stronger scaffold- and protein-homology-separated evaluations would support the generalization claims; comparisons with previously reported structure-based baselines should also be clearly distinguished from uniformly rerun experiments.
-
The evidence for describing cpd60 as an HCAR1 antagonist is insufficiently clear. HCAR1 is mainly Gi/o-coupled, and receptor activation commonly reduces forskolin-stimulated cAMP. Appendix E does not clearly describe an agonist-challenge experiment using lactate or CHBA, so reduced cAMP alone does not establish antagonism. Please clarify the direction of the reported inhibition, include agonist-challenge and receptor-specificity controls, and report independent biological replicates beyond the three technical replicates. The GloSensor and HTRF IC50 values (14.7 ± 1.2 and 2.5 ± 1.2 μM) indicate assay-dependent functional potency, not directly measured high binding affinity; the phrase “high-affinity antagonist” should be supported or softened. The related HCAR1 screening study by Fan et al. (2026) should also be discussed without conflating its target-specific approach with ReqVS's sequence-only setting.
-
The prospective HCAR1 results do not isolate the contribution of ReqVS. cpd60 was originally ranked 552nd and became first only after expert filtering for carboxyl and hydroxyl groups. Only one of the two filtered candidates was tested, alongside seven manually selected compounds, and the prospective screen used six rather than eight models. Consequently, the eight-compound rank–inhibition correlation cannot establish an unbiased top-K hit rate. Please report score-only and entropy-regularized rankings under the actual six-model configuration and distinguish the effects of computational ranking from expert selection.
References
- McGibbon, M., et al. (2023). SCORCH: Improving structure-based virtual screening with machine learning classifiers, data augmentation, and uncertainty estimation. Journal of Advanced Research, 46, 135–147. https://doi.org/10.1016/j.jare.2022.07.001
- Chen, L., et al. (2025). SCORCH2: A Generalized Heterogeneous Consensus Model for High-Enrichment Interaction-Based Virtual Screening. Advanced Science, 12(42), e08318. https://doi.org/10.1002/advs.202508318
- Moshawih, S., et al. (2024). Consensus holistic virtual screening for drug discovery: a novel machine learning model approach. Journal of Cheminformatics, 16, 62. https://doi.org/10.1186/s13321-024-00855-8
- Chen, L., et al. (2019). Hidden bias in the DUD-E dataset leads to misleading performance of deep learning in structure-based virtual screening. PLOS ONE, 14(8), e0220113. https://doi.org/10.1371/journal.pone.0220113
- Lind, S., et al. (2026). Profiling of HCAR1 signalling reveals Gαi/o and Gαs activation without β-arrestin recruitment and the discovery of an allosteric agonist. British Journal of Pharmacology, 183(10), 2219–2235. https://doi.org/10.1111/bph.70312
- Fan, M., et al. (2026). HCAR1 antagonist screening based on boundary-selected negative sampling strategy and multi-level graph neural network. Computer Methods and Programs in Biomedicine, 278, 109262. https://doi.org/10.1016/j.cmpb.2026.109262
Questions for authors
- What specific new algorithmic principle does Eq. (8) introduce relative to standard ensemble predictive entropy, and what evidence connects it to uncertainty from latent structural configurations?
- Are the modest improvements over ReqVS-ES statistically robust? Would ensemble predictive entropy, disagreement-based uncertainty, or simple consensus rankings achieve similar results?
- How are the DTA normalization-derived values calibrated as probabilities? How sensitive are entropy scores and top-K rankings to library composition and model weighting?
- What agonist-challenge and receptor-specific experiments demonstrate that cpd60 antagonizes HCAR1 instead of activating Gi-associated signaling or interfering with the cAMP assay?
- How much of the prospective ranking gain remains without expert-guided filtering, and how does the claim of novelty relate to Fan et al. (2026)?
13 comments
Response 2
Comment 2:
“W2: Equations (7)–(8) average the individual models' predictive entropies ... rather than measuring compound-specific disagreement ... Please compare the proposed estimator with ensemble predictive entropy, prediction variance, and mutual-information/Jensen–Shannon disagreement...” & “Q2: Are the modest improvements over ReqVS-ES statistically robust? Would ensemble predictive entropy, disagreement-based uncertainty, or simple consensus rankings achieve similar results?”
**Re:**To better address the reviewer's concerns, we evaluated alternative ranking and uncertainty-penalty strategies. Keeping the remaining screening settings fixed, we compared the proposed interaction-entropy penalty with no penalty, consensus ranking (rank averaging), and penalties based on Jensen–Shannon divergence (JSD), prediction variance, and absolute deviation. In these experiments, consensus ranking and disagreement-based penalties reduced screening performance, whereas ReqVS's interaction-entropy penalty consistently improved the reported screening metrics.
| Method | DUD-E EF0.1% | DUD-E EF0.5% | DUD-E EF1% | DUD-E BEDROC | DEKOIS2.0 EF0.1% | DEKOIS2.0 EF0.5% | DEKOIS2.0 EF1% | DEKOIS2.0 BEDROC |
|---|---|---|---|---|---|---|---|---|
| No penalty | 41.92 | 33.75 | 28.11 | 0.4543 | 17.22 | 18.18 | 18.12 | 0.5272 |
| Consensus ranking | 27.42 | 22.16 | 18.07 | 0.3072 | 13.78 | 13.40 | 12.18 | 0.3626 |
| JSD | 40.28 | 32.92 | 27.34 | 0.4453 | 17.22 | 17.54 | 17.61 | 0.5164 |
| Variance | 41.53 | 33.47 | 27.93 | 0.4514 | 16.84 | 18.12 | 17.92 | 0.5234 |
| Absolute deviation | 41.29 | 33.15 | 27.48 | 0.4475 | 17.22 | 18.12 | 17.67 | 0.5207 |
| ReqVS | 42.65 | 34.29 | 28.58 | 0.4586 | 18.37 | 18.69 | 18.69 | 0.5331 |
We would like to clarify that interaction entropy is not intended to penalize disagreement between models. Instead, it penalizes the uncertainty of each individual model’s prediction for a given protein–ligand pair, as defined in Eqs. (7)–(8) (L201–215). Intuitively, for predictions with the same ensemble mean, interaction entropy penalizes ambiguous individual predictions near 0.5 while favoring more confident predictions of activity or inactivity near 1 or 0, respectively.
We will add results and discussion of interaction entropy to the appendix in the revised manuscript.
Response 3
Comment 3:
“W3: The incremental benefit of entropy regularization is modest ... while AUROC slightly decreases ... Please provide paired target-level significance tests or confidence intervals, sensitivity analysis ... and evidence that entropy meaningfully improves the top-ranked candidate list.”
Re: We thank the reviewer for raising this concern about detail. We want to respectfully clarify that the 0.001 decrease in AUROC more closely reflects global ranking performance, while virtual screening aims to filter a small number of molecules from a large chemical library. In this scenario, the EF and BEDROC (raise EF0.1% by 1.01 and BEDROC by 0.005 in DUD-E) can more directly point out the improvement in the early recognition of ReqVS [1].
To better illustrate the contribution of interaction entropy, we first performed paired target-level comparisons between ReqVS and ReqVS-ES and estimated 95% confidence intervals using 50,000 paired bootstrap resamples of targets. The mean EF0.1% gains were 1.007 [95% CI: 0.217, 1.868] on DUD-E and 1.531 [0.383, 3.062] on DEKOIS2, with maximum target-level gains of 18.76 and 31.00, respectively.
| Benchmark (targets) | ReqVS-ES EF0.1% | ReqVS EF0.1% | Mean gain [95% CI] | Maximum target-level gain |
|---|---|---|---|---|
| DUD-E (102) | 41.64 | 42.65 | +1.01 [0.22, 1.87] | +18.76 |
| DEKOIS2 (81) | 16.84 | 18.37 | +1.53 [0.38, 3.06] | +31.00 |
Second, we added a sensitivity analysis for and compared the evaluation metrics. For , interaction entropy can robustly improve EF0.1% and EF0.5%, indicating that the gains are not restricted to our chosen value.
| λ | DUD-E EF0.1% | DUD-E EF0.5% | DUD-E EF1% | DUD-E BEDROC | DEKOIS2.0 EF0.1% | DEKOIS2.0 EF0.5% | DEKOIS2.0 EF1% | DEKOIS2.0 BEDROC |
|---|---|---|---|---|---|---|---|---|
| 0 | 41.92 | 33.75 | 28.10 | 0.4542 | 17.22 | 18.17 | 18.11 | 0.5272 |
| 0.05 | 42.12 | 33.80 | 28.18 | 0.4557 | 17.98 | 18.43 | 18.24 | 0.5296 |
| 0.1 | 42.12 | 34.00 | 28.37 | 0.4569 | 18.37 | 18.43 | 18.33 | 0.5313 |
| 0.2 | 42.65 | 34.28 | 28.57 | 0.4586 | 18.37 | 18.69 | 18.68 | 0.5330 |
| 0.4 | 42.51 | 34.50 | 28.63 | 0.4582 | 17.60 | 19.14 | 18.62 | 0.5322 |
| 0.8 | 42.64 | 34.32 | 28.00 | 0.4347 | 18.37 | 19.45 | 18.59 | 0.5157 |
| 1 | 40.95 | 31.31 | 22.85 | 0.3294 | 17.22 | 19.19 | 17.47 | 0.4613 |
Finally, to quantify the benefit of interaction entropy for the top-ranked candidate lists, we held other settings fixed, comparing only the absence or presence of the entropy penalty ( versus ). For each target, we selected the same number of top-ranked candidates under both settings and counted the known actives retrieved. These improvements were obtained without increasing the number of selected candidates, showing better recovery of known actives within the same selection budget with the proposed interaction entropy.
| Benchmark | Total candidates / actives / selected candidates | Per-target budget | Actives retrieved without entropy | Actives retrieved with entropy |
|---|---|---|---|---|
| DUD-E | 1,434,015 / 22,805 / 1,381 | Top-0.1% | 1,033 | 1,055 |
| DUD-E | 1,434,015 / 22,805 / 14,290 | Top-1% | 7,564 | 7,723 |
| DEKOIS2 | 100,408 / 3,239 / 486 | Top-0.5% | 285 | 293 |
| DEKOIS2 | 100,408 / 3,239 / 972 | Top-1% | 568 | 586 |
[1]Truchon J F, Bayly C I. Evaluating virtual screening methods: good and bad metrics for the “early recognition” problem[J]. Journal of chemical information and modeling, 2007, 47(2): 488-508.
Response 1
Thank you for your careful review and constructive feedback. We hope our responses have addressed your concerns and would be happy to clarify any remaining points. We will provide a revised manuscript shortly, incorporating the changes outlined in our responses.
Comment 1:
Re: We thank the reviewer for drawing our attention to these related studies. SCORCH combines predictors based on structural features and uses the standard deviation of model outputs to assess confidence. SCORCH2 further introduces weighted consensus scoring between two XGBoost models based on three-dimensional interaction features. Moshawih et al. integrate multiple screening scores and determine their weights based on training and validation performance. These methods require protein structures and pocket information, as well as molecular docking, which incurs additional computational time. These studies have advanced ensemble frameworks, and we will add a discussion of them to the Related Work section in the revised manuscript.
We would like to respectfully clarify that our methodological contributions include both the formulation of interaction entropy for sequence-based prediction and the unified integration of heterogeneous DTI and DTA models, as detailed below:
Interaction entropy. The interaction entropy is tailored to sequence-based models and motivated by an analysis of their underlying inference process. During inference, these models do not explicitly receive the three-dimensional complex conformation, binding pose, or intermolecular contact information for a given protein–ligand pair. These structural interactions are crucial to understanding molecular recognition and protein function [1], and predicting them is a central objective of approaches such as AlphaFold 3. We use mutual information to quantify the difference in predictive uncertainty with and without access to this interaction information, yielding a decomposition of predictive uncertainty into sequence-induced interaction uncertainty and intrinsic noise. As shown in Eq. (6), sequence-based models must implicitly account for possible structural interactions, giving rise to uncertainty. Based on this formulation, ReqVS uses Shannon entropy as a practical surrogate for this uncertainty (Eq. (8)).
Unified integration of DTI and DTA models. Our framework and consensus scoring scheme integrate heterogeneous DTI and DTA models independently of their underlying architectures. These model categories differ in both training objectives and output spaces: DTI models predict interaction propensity, whereas DTA models estimate continuous binding affinity. ReqVS brings their outputs onto a common numerical scale and combines information from both prediction tasks to improve candidate prioritization.
To clearly illustrate the robust gains in virtual screening performance provided by interaction entropy,we evaluated 53 combinations on DUD-E containing at least two DTI and two DTA models selected from the eight constituent models. ReqVS achieved higher EF0.1% than ReqVS-ES in 49 of these combinations. The balanced configurations—2 + 2 , 3 + 3 , and 4 + 4 —contain 36, 16, and 1 combinations, respectively.
[1]Salentin S, Schreiber S, Haupt V J, et al. PLIP: fully automated protein–ligand interaction profiler[J]. Nucleic acids research, 2015, 43(W1): W443-W447.