- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces StereoVSE, a multimodal framework for predicting enzyme–substrate specificity and prioritizing candidate specificity-determining residues. It combines pretrained protein representations, molecular features, and docking-derived pocket–substrate structures. A coupled vector–scalar graph encoder captures directional geometric information, while bidirectional residue–atom cross-attention learns substrate-conditioned interaction representations. Experiments on halogenase and glycosyltransferase benchmarks report improvements over six baselines, supported by component ablations and additional halogenase cold-start evaluations. Residue-level analyses compare attention-derived rankings with evolutionary conservation and examine prediction changes under computational mutations. Overall, the framework demonstrates promising predictive performance, although broader generalization, architectural novelty, and biological interpretation require stronger independent validation and controlled comparisons.
Critical strengths
- This paper reports consistent improvements over six baselines on both benchmarks, including halogenase AUPR increasing from 0.8664 to 0.9112 and glycosyltransferase MCC increasing from 0.6198 to 0.6813.
- This paper combines complementary sequence, chemical, and structural representations within a coherent architecture, with ablations on both datasets supporting the contributions of the structural encoder and interaction module.
- This paper extends prediction to substrate-conditioned residue prioritization and appropriately acknowledges that attention scores represent attribution signals and that its mutation analysis holds the structural context fixed.
Critical weaknesses
- This paper provides insufficient evidence of robust generalization: cold-start experiments cover only halogenases, identity-based partitions do not exclude close homologs or similar molecular scaffolds, and “pair-cold” evaluation remains a seen-enzyme, seen-substrate setting rather than a jointly unseen setting.
- This paper does not sufficiently isolate its architectural novelty from established vector–scalar message passing and bidirectional cross-attention. Matched-encoder, matched-capacity comparisons against alternative geometric encoders and simpler interaction modules would clarify whether the gains arise from the proposed mechanisms.
- This paper provides limited independent validation of residue attribution. Spatial proximity, disagreement with conservation, and prediction changes from the same model do not establish biological specificity determinants; quantitative evaluation against experimentally characterized mutations or prospective biochemical validation is needed.
Questions for authors
NA
11 comments
Q2: Disentangling architectural novelty from existing geometric and attention modules
A2: Clarification of architectural novelty via controlled substitution ablations
We sincerely thank the reviewer for this insightful and constructive comment regarding the architectural novelty of StereoVSE and the need to distinguish its contributions from established geometric message-passing and cross-attention mechanisms.
First, we would like to clarify that our contribution is not the introduction of vector–scalar message passing or bidirectional cross-attention as entirely new operations. Rather, StereoVSE integrates these mechanisms into a unified framework for enzyme–substrate specificity prediction. Specifically, the framework constructs a joint three-dimensional pocket–substrate graph that explicitly represents intermolecular spatial relationships, employs coupled scalar and directional vector representations to encode the local binding geometry, and combines these structural representations with bidirectional residue–atom interaction features and global chemical information for specificity prediction. This design aims to capture complementary geometric and sequence–chemical information within a single predictive framework.
To better isolate the architectural contributions, we have conducted additional controlled ablation experiments in which individual components are replaced with alternative implementations while the remaining components and pretrained representations of StereoVSE are retained.
In particular, we considered three architectural alternatives (Table R10):
1.Intra-molecular 3D: Removing cross-molecular pocket–substrate edges while retaining the intramolecular three-dimensional information.
2.BRIA → dual cross-attention: Replacing BRIA with EZSpecificity's dual cross-attention interaction module.
3.V–S → EGNN: Replacing the coupled Vector–Scalar geometric encoder with EZSpecificity's distance-based EGNN.
Table R10: Ablation results on the halogenase benchmark. The best results are shown in bold.
| Variant | AUC | AUPR | F1 | ACC | MCC |
|---|---|---|---|---|---|
| intra-only 3D | 0.9605 | 0.8723 | 0.7870 | 0.9098 | 0.7298 |
| BRIA$\rightarrow$dual cross-atten | 0.9657 | 0.8817 | 0.7936 | 0.9171 | 0.7431 |
| V-S$\rightarrow$EGNN | 0.9619 | 0.8694 | 0.7775 | 0.9125 | 0.7260 |
| StereoVSE | 0.9728 | 0.9112 | 0.8138 | 0.9238 | 0.7666 |
The results consistently favor the complete StereoVSE architecture on both benchmarks. On the halogenase dataset, the full model achieves an AUPR of 0.9112, whereas replacing BRIA with dual cross-attention reduces AUPR to 0.8817, replacing the Vector–Scalar encoder with EGNN reduces it to 0.8694, and removing cross-molecular structural edges reduces it to 0.8723. Similarly, on the glycosyltransferase dataset (Table R11), the full model achieves an AUPR of 0.7338, compared with 0.7036 for the dual cross-attention variant, 0.7010 for the EGNN variant, and 0.6973 for the intra-molecular 3D variant.
Table R11: Architectural ablation results on the glycosyltransferase benchmark. The best results are shown in bold.
| Ablation Module | AUC$\uparrow$ | AUPR$\uparrow$ | F1$\uparrow$ | ACC$\uparrow$ | MCC$\uparrow$ |
|---|---|---|---|---|---|
| intra-only 3D | 0.9259 | 0.6973 | 0.7212 | 0.8712 | 0.6410 |
| BRIA$\rightarrow$dual cross-atten | 0.9267 | 0.7036 | 0.7275 | 0.8653 | 0.6506 |
| V-S$\rightarrow$EGNN | 0.9290 | 0.7010 | 0.7375 | 0.8719 | 0.6632 |
| StereoVSE | 0.9364 | 0.7338 | 0.7500 | 0.8762 | 0.6813 |
These comparisons are conducted within the same overall framework, with the remaining pretrained representations and modeling components preserved. Therefore, the observed performance differences provide evidence that the proposed architectural choices contribute to predictive performance beyond the selection of pretrained protein and molecular representations.
We sincerely appreciate the reviewer's valuable suggestion, which has helped us clarify the architectural contribution and the interpretation of our experimental evidence. The additional architectural ablation results are presented in Section 4.4 and Appendix D of the revised manuscript.
R3: Independent experimental validation for residue attribution is required
R3: Retrospective Mutagenesis Validation for Residue Prioritization
We sincerely thank the reviewer for this insightful and constructive suggestion regarding the independent experimental validation of residue rankings.
To address this concern, we conducted an additional retrospective evaluation using an independently published mutagenesis screen of the RebH variant 3-LSR [1]. In this study, 13 positions near the substrate-binding pocket were experimentally screened for improved bromination activity toward indole-6-carboxylic acid. Among these positions, four (Y455, F465, W466, and N470) were identified as activity-enhancing positions, each having at least one mutation that increased bromination activity by at least twofold relative to the parental RebH 3-LSR enzyme.
Table R11. Experimentally Identified Activity-Enhancing Mutations in RebH 3-LSR [1]
| Experimentally identified position | Activity-enhancing single mutants | Reported activity relative to 3-LSR |
|---|---|---|
| Y455 | Y455W, Y455C | ≥2-fold |
| F465 | F465K, F465N | ≥2-fold |
| W466 | W466I, W466Y | ≥2-fold |
| N470 | N470S, N470K | ≥2-fold |
We evaluated BRIA-derived residue rankings against these experimentally characterized outcomes. Specifically, using wild-type RebH, we calculated a substrate-pair differential attention score based on the absolute difference in residue-level BRIA attention between L-tryptophan and indole-6-carboxylic acid. All ranking methods were evaluated on the same 13 experimentally screened positions, with the four activity-enhancing positions treated as positives, ensuring a consistent comparison.
Notably, BRIA substrate-pair differential attention ranked all four experimentally identified activity-enhancing positions above the remaining nine screened positions, achieving an AUROC of 1.000. In comparison, global BRIA ranking achieved an AUROC of 0.722, minimum distance to bound L-tryptophan achieved 0.694, distance to the L-tryptophan carboxyl group achieved 0.528, and alignment-based sequence conservation achieved 0.222.
Table R12. Quantitative Comparison of Residue-Ranking Methods Against Experimental RebH Mutagenesis Data
| Ranking method | AUROC | Exact permutation p-value |
|---|---|---|
| BRIA substrate-pair differential attention | 1.000 | 0.0028 |
| Global BRIA ranking | 0.722 | 0.26 |
| Minimum distance to bound L-tryptophan | 0.694 | 0.33 |
| Distance to the L-tryptophan carboxyl group | 0.528 | 0.94 |
| Alignment-based sequence conservation | 0.222 | 0.15 |
The two-sided exact permutation test yielded a p-value of 0.0028 for the BRIA substrate-pair ranking, supporting the statistical significance of the observed separation within the 13 screened positions. Furthermore, when ranked across the complete 530-residue sequence, W466, N470, Y455, and F465 occupied positions 37, 39, 62, and 64, respectively.
To assess potential training-data overlap, we repeated the analysis after excluding N470, as an N470S variant was included in the training data. The BRIA substrate-pair ranking maintained an AUROC of 1.000 across the remaining 12 positions (two-sided exact permutation p = 0.0091), indicating that the observed result was not solely driven by N470.
Ref.
[1] Sana B, Ho T, Kannan S, Ke D, Li EHY, Seayad J, Verma CS, Duong HA, Ghadessy FJ. Engineered RebH Halogenase Variants Demonstrating a Specificity Switch from Tryptophan towards Novel Indole Compounds. ChemBioChem. 2021;22(18):2791–2798. doi: 10.1002/cbic.202100210. PMID: 34240527.
Q1: Insufficient generalization evidence from limited cold-start splits
R1: Expanded cold-start evaluations and adoption of double-cold protocol for generalization assessment
We sincerely thank the reviewer for this insightful and constructive comment regarding the generalization capability of StereoVSE.
Regarding the evaluation scope, we have substantially extended our cold-start experiments beyond the halogenase benchmark. Specifically, we conducted three complementary cold-start evaluations—enzyme-cold, substrate-cold, and double-cold—on the glycosyltransferase benchmark as well. These settings assess the ability of StereoVSE to generalize to unseen enzyme identities, unseen substrate identities, and simultaneously unseen enzyme–substrate identities, respectively. The additional glycosyltransferase results are provided in Table R8 of the revised manuscript.
Regarding the limitations of the original pair-cold setting, we replaced the original pair-cold protocol with a more stringent enzyme–substrate double-cold setting, in which both the enzyme and substrate identities appearing in the test set are strictly disjoint from those in the training set.
Regarding the experimental results (Table R9), StereoVSE consistently maintains its performance advantages under the expanded cold-start evaluations. On the halogenase benchmark, StereoVSE achieves an AUPR of 0.762 under the double-cold setting, compared with 0.620 for EZSpecificity. Similarly, on the glycosyltransferase benchmark (Table R8), StereoVSE achieves an AUPR of 0.529, compared with 0.506 for EZSpecificity. Furthermore, StereoVSE outperforms the evaluated baselines under the enzyme-cold and substrate-cold settings on both benchmarks. These additional results provide stronger evidence that the predictive improvements are not restricted to previously observed enzyme or substrate identities.
We sincerely appreciate the reviewer's valuable suggestions. The revised cold-start protocols and additional results are presented in Section 4.3 and Appendix E.1 of the revised manuscript. The concerns regarding homologs and similar molecular scaffolds will be addressed in Response R4 (Rigorous cold-start evaluations controlling for homologous enzymes and substrate scaffold similarity are ongoing, and we will report these results once they are available) .