- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose ALLOMIT, which adapts pretrained virtual-screening methods on allosteric activity data. It first uses FPocket to identify several pockets and encodes each with a pocket encoder. Then it uses the CSPA module to sparsely aggregate the pocket embeddings to obtain a final score. The authors also construct AlloVSBench for training and evaluation. After fine-tuning, AlloMIT outperforms both the pretrained HypSeek and fine-tuned HypSeek.
Critical strengths
The experiments include backbone comparisons including , , , , , and their fine-tuned versions. It gives fair comparisons and proves that AlloMIT is better at leveraging limited activity data than the baselines. They also provide detailed ablations on the architectural choices.
Critical weaknesses
- The supervision does not identify the correct pocket and an incorrect pocket assignment can satisfy the training objective. The authors did not prove that successful retrieval depends on the experimentally supported pocket. The authors did not explain the motivation behind the designs behind section 2.3. The authors report sequence-only results, but do not establish why adding potentially unreliable pocket candidates improves over a strong ESM-based sequence-only model.
- The allosteric test set is extremely small with only 25 allosteric positive pairs across 19 targets. Also, separating exact UniProt identities does not establish that the test pockets are structurally unfamiliar. Since reference structures and sites are available, the authors should measure local pocket sequence/structural similarity and ligand similarity against both adaptation data and relevant supervised backbone training data.
Questions for authors
- Does the method depend on the correct site?
- How were checkpoints selected, and how stable are the results?
- Could the authors provide a complete reproducibility package?
- Could the authors perform more case studies as in 4.5. For example, 8QYQ (chain A, H), 8U6B (chain A, C)?
5 comments
Q1 (follow-up). The answer restates that the allosteric site is not an input; that was not the question. We asked whether the pocket that drives a ligand's score corresponds to the experimentally supported site, and whether performance survives when it does not. The K-versus-coverage table cannot resolve this, because every bag contains both correct and incorrect pockets, so coverage and pocket correctness are confounded. Please report, per test target, whether the top-scoring pocket or the pocket producing the best ligand rank overlaps the annotated site, and compare performance when scoring is restricted to the annotated pocket versus randomly selected pockets of matched size.
Q4 (follow-up). We did not ask for library-scale screening on these two structures. 8QYQ and 8U6B were suggested because each contains a defined allosteric ligand bound at a defined site. Please test that pair directly. Report (i) the score and rank of that ligand across the candidate pockets, (ii) the CSPA weight per pocket in the style of Figure 9, and (iii) whether the highest-weighted pocket corresponds to the crystallographic binding site, with contact residues compared. Decoys should come from a fixed, size-matched set rather than a 20-million-compound library.
Response to Reviewer d4Sb
Thank you for your insightful comments. We have clarified the role of candidate-pocket coverage and checkpoint selection, provided stability analyses and reproducibility information, and extended the case studies to the suggested targets. To make our revisions and supporting data easier to inspect, we provide an anonymous repository here.
Q1. Does the method depend on the correct site?
A1. AlloMIT does not require the correct allosteric site to be specified in advance. It generates a bag of candidate pockets and scores each ligand against these candidates. Moreover, the table below suggests that screening performance is more closely related to target coverage than to precise pocket–site matching.
Q2. How were checkpoints selected, and how stable are the results?
A2. Given the scarcity of allosteric adaptation data and the absence of experimentally verified inactive compounds in the training set, we did not hold out a separate validation set and instead selected AlloMIT checkpoints using training loss. As shown in Figure 1, we selected the checkpoint with the lowest training loss independently for each stage. Test-set performance was strictly excluded from both checkpoint selection decisions.
We assessed result stability across random seeds and sensitivity to the inclusion of individual test targets. Across three random seeds, allosteric BEDROC was 23.19% ± 0.11%. LOO denotes leave-one-target-out analysis, where each test target is removed in turn and the mean gain is recomputed on the remaining targets. As shown in the table below, AlloMIT retained positive mean gains over HypSeek for both evaluation panels and both metrics.
Q3. Could the authors provide a complete reproducibility package?
A3. To facilitate reproducibility, we provide an anonymous repository containing the code and usage instructions. The training data will be publicly released upon acceptance of the paper.
Q4. Could the authors perform more case studies as in 4.5. For example, 8QYQ (chain A, H), 8U6B (chain A, C)?
A4. Thank you for suggesting these cases. We extended the case study to 8QYQ and 8U6B and report them alongside AKT1. None of these three cases was included in the training set. We evaluated these targets using the screening library from Section 4.5 and a commercial screening library containing 20 million compound records. Across all three targets and both screening libraries, AlloMIT consistently ranks the allosteric ligands ahead of HypSeek.
Screening library from Section 4.5
Commercial screening library