- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Ctrl-P, a preference alignment framework for multi-scenario protein design. Instead of training on scarce paired multi-context data, Ctrl-P coordinates multiple single-scenario generators by exchanging ProteinMPNN-derived sequence preferences during inference. Through an iterative negotiation process, it designs sequences that satisfy multiple objectives, achieving strong performance in multi-target binder design, motif switching, and multi-target viral binder design.
Critical strengths
- The paper addresses an important but challenging problem in protein design: generating proteins that satisfy multiple contextual objectives. The proposed preference alignment framework provides a practical alternative to directly training multi-scenario models, which is difficult due to the scarcity of paired multi-context structural data.
- The method demonstrates substantial improvements over existing baselines on challenging benchmarks, especially for dissimilar target pairs in multi-target binder design.
- The negotiation procedure, temperature-controlled preference strength, and preference population representation are well motivated and provide a reasonable mechanism for balancing exploration and exploitation during multi-scenario optimization.
Critical weaknesses
- The statement “we cannot always compare refolded structures to model-generated structures because of multiple contexts” is not convincing. In fact, for each designed sequence, one can obtain context-specific predicted complexes (e.g., design–target A and design–target B) and compare these structures against the corresponding model-generated structures. The current evaluation appears to rely heavily on confidence metrics such as pLDDT, which may not adequately capture whether the designed sequence adopts the intended conformation in each context. A direct structural comparison (e.g., backbone RMSD or interface-level metrics) between predicted and generated complexes would provide a more rigorous assessment.
- I question whether the preference-conditioned generator is genuinely performing generative design or whether it is functioning primarily as a folding model conditioned on sequence preferences. The ProteinMPNN-generated sequence populations provide an MSA-like representation that may already encode substantial structural information. Consequently, the role of the generative model during iterative sampling appears unclear: beyond providing an initial structural proposal, subsequent rounds seem to mainly optimize consistency between the structure and the supplied sequence preferences. It would therefore be important to clarify what additional design capability the generative model provides over a folding model with similar sequence-conditioning inputs.
- The benchmark lacks structural visualization and qualitative analysis. Since the benchmark is newly constructed and evaluation relies heavily on folding-model predictions, it is important to show representative target structures and successful designs. Folding models may have biases, particularly for challenging multi-scenario tasks, and confidence-based metrics alone may not fully reflect whether the generated structures are physically plausible. Structural inspection would provide additional evidence supporting the reported improvements.
Questions for authors
The definition of "scenario" can be more general. Can Ctrl-P handle multi-ligand scenarios where different ligands induce different states, and can it design protein–protein interaction-driven motif switches where different binding partners trigger distinct conformational states? Can it design (with protein target A, without protein target A)?
4 comments
Thank you for your responses. The result of "inverse folding only" is interesting and all my "critical weaknesses" are resolved. I’m still curious about the questions I raised in the review and would appreciate your thoughts on them.
Thank you, we are glad to have been able to address your concerns, and apologies for forgetting to respond to your question. Yes, the intention is for a “scenario” to be very general. As design models get better, we expect them to handle an increasing number of individual scenarios, such as protein binding, small molecule binding, nucleic acid binding, enzyme motif scaffolding, protein binding + motif scaffolding, etc. The strength of our approach is then that we can finetune any of these models to condition on preferences and thereby combine their scenarios. The models we coordinate could also be completely different, as long as they share the same preference conditioning interface. Here we restricted ourselves to bivalent binders and allosteric switches because of the scenarios supported by the models we fine-tuned. For example, the protein-protein interaction driven motif switches would require a model that can condition on a target and a motif at the same time, yet Complexa only supports target conditioning (we had to finetune a model for ligand binding + motif scaffolding for the allosteric switch experiments). Nevertheless, the fact that the same approach already works for both bivalent binders and allosteric switches gives us evidence that it can work for other combinations as well. Thus, as more generalist models become available, we aim to support these more diverse tasks as you suggested.
Thank you. All my concerns and questions are resolved. I will maintain my original positive score.
Response to review
Thank you for your review. Please find below our response to your question and the concerns you raised.
On the self-consistency RMSD criterion The table below shows the number of challenging tasks solved for Ctrl-P and Proteina-Complexa with beam search when including scRMSD as an additional criterion. Note that, in our negotiation algorithm, at any given round a design is produced by a generative model conditioned on a single target. Thus, for each design, there is only an associated structure for one of the two targets, and this is the structure we use for the self-consistency metric. The same also applies for the beam search baseline. The confidence metrics are evaluated against both targets, since they only depend on the sequence of the design. Overall, we see that the scRMSD does not tend to gate successes compared to the other confidence metrics. We have also added these results into Appendix D.12 of the paper. Given these results, we would opt to keep our current success criteria for the main results. Metrics that only depend on the design sequence are more general and allow us to evaluate against more baselines using the exact same criteria. For example, BindCraft does not have any generated structures to compute scRMSD against since it directly optimizes the design sequence. | | No gate | scRMSD < 3 Å | scRMSD < 2 Å | scRMSD < 1.5 Å | |---|---|---|---|---| | Ctrl-P | 8 | 8 | 8 | 8 | | Proteina-Complexa + beam search | 2 | 2 | 2 | 2 |
On the advantage over a folding model Yes, we have evidence that the generative model adds value beyond the folding model. We measured successes on one of the hard multitarget tasks while allowing the structure to change more or less between rounds. As shown in the table below, successes peak when we allow the structure to change, as opposed to keeping the structure fixed and only redesigning the sequence (k controls the amount of redesign, where k=0 corresponds to fully redesigning the structure, and k=100 corresponds to no redesign and only inverse folding the sequence). In more detail, recall that we seed the design model's generation with the output of a folding model. This is because the design model itself is not good at folding. As described in more detail in Appendix D.3, we noise the folding model prediction, and use the generative model to denoise it. The parameter k here refers to the denoising step that we renoise to, so k=0 means fully renoising and k=100 means no renoising at all. We see that, if we only let the model do inverse folding, we find far fewer successes than if we allow it to change the structure. We have also added these results to Appendix D.3 of the paper. | k | 8 | 16 | 24 | 32 | 48 | 64 | 80 | 100 | |---|---|---|---|---|---|---|---|---| | Unique successes (Ctrl-P, DIS04) | 6 | 17 | 35 | 4 | 3 | 1 | 2 | 3 |
On structural visualizations We have added several visualizations of target pairs and successful designs to Appendix D.13 of the paper. Overall we see that while the similar targets share binding interfaces, the dissimilar targets show limited alignment. The successful dual binders largely maintain their backbone geometry when binding to either target. Figure 16 shows the binding interfaces, where we can see that the binders use different residues to engage the dissimilar targets, while the similar targets tend to engage more of the same parts of the binder epitopes and residues.
Thank you again for your review, we appreciate the time and effort it takes. Please let us know if we have sufficiently addressed your concerns or if you have any follow-up questions.