- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes MevoCraft, a method that injects state-condition motif MSA information into each forward pass of a gradient-based protein sequence optimization pipeline. The method outperforms SwitchCraft, another gradient-based approach without MSA information, across several design tasks.
Critical strengths
- The proposed method is plausible, as using MSA information is important in protein design tasks. It is interesting to see how such information can be injected into continuous states during the optimization process without exhausting search at each step. This approach, to my knowledge, is novel and worth further exploration in future work.
- The paper is clearly written and easy to follow.
Critical weaknesses
- My main concern is about the experiments. The paper only compares with SwitchCraft, which shares most part of the algorithm, and lacks comparisons with other protein design frameworks that incorporate MSAs into discrete states.
- The experimental results do not include any statistical tests. Table 2 shows that using full MSA at evaluation only can perform well and is slightly behind the proposed approach.
Questions for authors
- Table 1 lacks diversity and novelty metrics. Line 258-259 state that MSAs may bias structure predictors towards particular conformations. Without novelty and diversity, it is unclear how this bias affects the obtained designs?
3 comments
Q1: Add diversity and novelty metrics
R3: We thank the reviewer for this valuable suggestion. To investigate whether MSA conditioning restricts the diversity and novelty of generated proteins, we conducted additional evaluations comparing MevoCraft and SwitchCraft across all positive- and negative-allostery cases.
Sequence Diversity. For each motif–ligand case, we measure sequence diversity among the 100 generated designs using pairwise global Needleman–Wunsch alignment. For a pair of sequences , let denote the number of alignment columns containing identical amino acids, and the total alignment length, including gap-containing columns. We define pairwise identity and sequence diversity as
D=1-\frac{2}{N(N-1)}\sum_{i<j}\mathrm{Identity}(x_i,x_j).$$ **Sequence Novelty.** To assess novelty relative to known natural proteins, we search each designed sequence against Swiss-Prot using MMseqs2 with two search iterations, up to 2,500 hits per query, and an E-value threshold of $10^{-3}$. For a designed sequence $x$, novelty is defined as $$\mathrm{Novelty}(x)=1-\max_{h\in\mathcal D}\mathrm{Identity}(x,h),$$ where $\mathcal D$ denotes the retained Swiss-Prot hits and identity is calculated over the MMseqs2 alignment. If no hit satisfies the search criteria, novelty is assigned a value of 1. **Results.** The following table summarizes the average diversity and novelty across all positive- and negative-allostery cases. | Task | Metric | MevoCraft | SwitchCraft | | --- | --- | ---: | ---: | | Positive allostery | Diversity ↑ | 0.612 | 0.610 | | Negative allostery | Diversity ↑ | 0.609 | 0.608 | | Positive allostery | Novelty ↑ | 0.722 | 0.723 | | Negative allostery | Novelty ↑ | 0.718 | 0.713 | The results show that **MevoCraft maintains comparable sequence diversity and novelty to SwitchCraft**, despite achieving substantially higher design success rates. This suggests that motif-scoped MSA conditioning can guide structure prediction toward desired motif conformations without substantially restricting the diversity or novelty of the resulting sequences. Furthermore, we computed the cross-method sequence similarity between designs generated by MevoCraft and SwitchCraft, obtaining an average similarity of approximately 0.385. This indicates that the two methods generate substantially different sequences even under the same design objectives, suggesting that MevoCraft does not merely reproduce the sequences explored by SwitchCraft. We have added these diversity and novelty evaluations into Sections B.4–B.6 of the revised appendix. Thanks again for your suggestion. --- We sincerely appreciate the reviewer's constructive feedback. We hope that these additional experiments and analyses adequately address your concerns and further strengthen your confidence in our work.Reply
Hi, thank you for your response and for conducting the additional experiments, which have addressed most of my concerns regarding the experimental results. I have therefore decided to raise my rating to 4.
We thank the reviewer for the positive assessment of our work and the constructive feedback. We are encouraged that the reviewer recognizes the novelty of our approach and its potential for further exploration. We address each of your concerns below.
W1: Limited comparisons with other protein design frameworks
R1: We thank the reviewer for raising this concern. To broaden our experimental evaluation, we extended MevoCraft to an alternative structure predictor (Protenix) and an additional protein design task (motif scaffolding), demonstrating its applicability beyond the original Boltz-based multistate design setting.
Generalization to Protenix. We integrated MevoCraft into Protenix and performed the full gradient-based sequence optimization using Protenix as the structure predictor. We also reproduced SwitchCraft using Protenix under the same optimization protocol for a fair comparison. The experiments cover four motifs and five conditioning ligands for both positive and negative allostery, with 100 independently optimized designs per case.
MevoCraft-P achieves 2.81× and 1.33× as many successful designs as SwitchCraft-P for positive and negative allostery, respectively. These results show that the proposed motif-scoped evolutionary conditioning strategy can be integrated into a different structure predictor. Detailed results are provided in Appendix B.8.
Generalization to Motif Scaffolding. We further extended MevoCraft to the motif-scaffolding task, evaluating five functional motifs with 100 designs per motif. For this task, we employ the motif loss and contact loss defined in our design specification, keeping the motif residues fixed while optimizing the surrounding scaffold residues to support the target motif geometry. A design is considered successful if its mean motif Cα RMSD across five structure samples is at most 1.0 Å and its mean pLDDT exceeds 0.7.
MevoCraft-MS improves the overall success rate by 10.2%, while achieving lower motif RMSD and higher pLDDT. These results show that the evolutionary conditioning strategy is also applicable to protein design tasks beyond multistate switching.
Detailed results are provided in Appendix B.7. Thanks again for your comment.
W2: Full MSA at evaluation is slightly behind MevoCraft and lack of statistical tests
R2: We thank the reviewer for this important comment. To further investigate whether motif MSA conditioning benefits sequence optimization beyond its effect on final evaluation, we extended the comparison between MevoCraft and its evaluation-only variant (MevoCraft-eval) from a single motif in Table 2 to 70 positive-allostery cases, 70 negative-allostery cases, and 30 motif-switching cases.
The results are summarized below:
MevoCraft consistently achieves higher success rates than MevoCraft-eval across all three tasks, with relative improvements of 26.0%, 23.7%, and 36.2%, respectively. To quantify the uncertainty of these improvements, we performed bootstrap analyses by resampling motifs, while keeping the five ligand cases within each resampled unit together. The results are summarized below:
These results further support the benefits of incorporating motif MSA conditioning during optimization rather than only at evaluation.
We have added the detailed per-case success-count differences and statistical analyses to Appendix C.1. Thanks again for your comment.