- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose a binding pose prediction benchmark for enzyme systems. Since enzyme–substrate complexes are often not static, experimentally determined crystal structures are limited. Rather than evaluating how well methods reproduce entire binding poses, the proposed benchmark evaluates how they capture interactions between pre-defined reacting atom pairs by measuring the distances between them. The authors first validate the distance criterion -- 1.4 x (r1 + r2) --against resolved crystal structures. Then, co-folding models are evaluated on enzyme–ligand systems with known reaction centers.
Critical strengths
The benchmark addresses the scarcity of experimentally resolved enzyme–substrate complexes by focusing on the proximity of reacting atom pairs rather than the reproduction of complete binding poses. This reaction-centered evaluation provides a targeted way to assess predicted enzyme–ligand structures. The authors also validate the proposed distance criterion against experimentally resolved crystal structures before applying it to co-folding predictions.
Critical weaknesses
- As the authors acknowledge, the current criteria consider distances between reacting atoms but omit their relative orientation. Molecular orientation is an important determinant of chemical reactivity: even when reacting atoms are sufficiently close, an unfavorable orientation may prevent the complex from satisfying the geometric requirements for reaching the transition state. Consequently, distance alone provides an incomplete assessment of whether a predicted binding pose is compatible with catalysis. I recommend incorporating reaction-specific angular criteria, at least for representative reaction types, and reporting their effects on pass rates and model rankings. The angular criteria implemented in PLIP [1] for non-covalent interactions could provide a useful implementation reference.
- The evaluation is limited to co-folding models. Because the benchmark assesses the geometric plausibility of predicted enzyme–ligand complexes, physics-based docking methods would provide important baselines. Moreover, Boltz2ESI [2] reports that EZSpecificity [3], which uses AF2 and Vina-based rigid docking, performs competitively with the co-folding-based Boltz2ESI approach. Including physics-based docking baselines alongside pocket-conditioned co-folding models would therefore strengthen the comparative evaluation.
References
- https://github.com/pharmai/plip
- Cheng, Xiwei, et al. "Boltz2ESI: Accurate Enzyme Specificity Prediction with Co-folding Foundation Model." ICLR 2026 GEM
- Cui, Haiyang, et al. "Enzyme specificity prediction using cross-attention graph neural networks." Nature 647.8090 (2025): 639-647.
Questions for authors
The main issues that could affect my assessment are already outlined in the Critical Weaknesses section. I am willing to raise the score based on the author's response.
3 comments
Thank you for your response and additional experiments. I'll increase my score to 3.
Thank you for your positive response. We appreciate your time and thoughtful feedback!
We thank the reviewer for the constructive comments. We have added the corresponding analyses and experiments to the revision (new appendix material is marked in blue), as summarized below.
We agree that NAC geometry depends on orientation as well as distance. We therefore classified reacting atom pairs in the core set by reaction type (Appendix B.5, Figure 14a). The three most frequent classes are proton transfer (55%), SN2-like substitution (20%), and nucleophilic addition (17%). Since hydrogen coordinates are often uncertain, we do not impose angular constraints on proton-transfer pairs. For SN2-like substitution and nucleophilic addition, we introduce angular criteria based on their expected reaction geometries, using broader ranges of 150°–180° and 90°–120°, respectively, to accommodate deviations in single predicted structures. As shown in Appendix B.5, Table 7, these additional constraints lower absolute success rates, but model rankings remain largely stable. The angle distributions of distance-passing Protenix-v2 Top-1 predictions also peak near the expected geometries (Appendix B.5, Figure 14b,c).
We agree that including a classical docking baseline strengthens the evaluation. We therefore evaluated AutoDock Vina on 75 core-set entries with a single substrate and no metal-dependent catalytic centers, for which Vina is applicable. We used the top-ranked Protenix-v2 protein structure as the receptor and centered the docking box on the predicted substrate position. Success was assessed using the same reacting-atom distance criterion as for the co-folding models. As shown in Appendix B.6, Table 8, co-folding models generally perform better without the PB-Valid filter. With this filter, Vina becomes competitive at Top-1, while Boltz-1x achieves the strongest Top-5 performance.
We thank the reviewer again for the careful reading and helpful suggestions.