- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces a new inference time method called RipZipGuide that samples from a “stability” tilted distribution of a base model. RipZipGuide operates by (1) gradually destabilizing the protein via gaussian noise, (2) discovers unstable links, and (3) modify these unstable links via mutation to have stronger bonds. They demonstrate the benefits of this method on both synthetic and real protein datasets.
Critical strengths
- The heuristic of gradual destabilization to identify weak links and then replacing those weak links with mutations that strength the structure is unique and interesting
- The use cases foor generating more stable proteins are important
Critical weaknesses
-
Writing is very hard to follow, especially the Methods section. For instance, Line 215-233 and Algorithm 1 circularly depend on one another to describe a single algorithm. Further, the description of the algorithm is very textually dense, making it hard to understand the brass tacks of the algorithm with many terms and verbs used without rigorous definition (e.g. “denoise” is used but what does this mean rigorously? Also, please see questions 3,4,5)
-
I appreciate the problem of developing more stable proteins that the authors attempt to tackle. However, the motivation for why the proposed method needs to be the way it is very unclear. Not only is RipZipGuide extremely complex and heuristic (many arbitrary choices of cutoffs and thresholds throughout Algorithm 1 and the Methods section), the necessity of this complexity is not well motivated. Why wouldn’t a much simpler algorithm work? For example, define a scalar quantity for “stability” and then use any off the shelf inference time sampler to sample from the base distribution tilted by this “stability.”
Questions for authors
- Is there a more rigorous or quantitative definition of stability? The reason I ask this is if there’s some way we can quantitatively compute “stability,” could we just leverage any off the shelf inference-time conditional sampling method for diffusion models (like BoltzGen) rather than the used heuristic of destabilization + patching
- What does “DPS” mean (used in subtitle of 3.1 without definition)?
- Diffusion models operate via a noise conditioned score function. After noising the protein structure in step 1 of Section 3.2, which noise level of the score network (sigma in Eq. 1) do you feed this noised protein structure into for denoising? The lowest noise level or from the beginning (aka highest noise level)?
- RE Line 208, the authors use this notion of “destabilization” and make the logical equivalency of thermal destabilization = gaussian noising. More realistically, isn’t destabilization better represented as raising the temperature, rather than some artificial gaussian noising process that does not correspond to anything physically meaningful?
- The authors refer to “accurate” unfolded pathways, what does "accurate" here mean?
4 comments
Response to Reviewer CEur Part 2
Which noise level is fed in?
Thank you for pointing out this lack of context.
The noise dynamically grows (starting from very low magnitudes) to keep up with the basin exploration. At a given RipGuide step, the next noise level is based on RMSD-s of a few (usually 10) of the latest-sampled disturbed structures from the original. We also highlight this in eq.6.; Supp Fig5 shows that any constant noise level does not allow "physical" exploration, while this dynamical scheme performs well.
- We use an intermediate, dynamically increasing noise level. This is now explained in more detail in point3 in Section 3.2. You can also find more details in Section C.1. We start with a very low level of noise, however, we must increase it to reach structures progressively further from the original structure. This progressive increase in the noise level should happen only after the region of the currently-explored basin (in conformation space) covered by the current level of noise fills-up. Thus, we dynamically tie (eq.6) the noise level to the RMSD-from-original of several (usually 10) latest structures.
- Such a complicated approach may look unnecessary, similar to other heuristics you noted above. However, the Toy-model shows (Supp Fig.5) that this dynamics allows finding nearby basins in the “physical” order: using a too-low constant noise level does not allow any nearby basins to be discovered, while too-high constant noising discovers all basins (within the noise level), but does so in random order, which violates the termination logic of RipGuide, which assumes that samples deviating further from the original are (mostly) sampled after all basins closer to the original are sampled and filled-up.
destabilization = Gaussian noising?
Thank you for pointing out this confusing point.
Gaussian noise on its own was not meant to be presented as analogous to thermal destabilization. Instead, it is part of the noise-denoise cycles used by RipGuide to generate progressively disturbed structures of the fixed protein sequence (Fig3(b,e)). We meant to communicate that these disturbed structures resemble a protein gradually unfolding from thermal destabilization.
The sentence at line 208 (since changed) referred to the whole metaD-like destabilization approach as thermal-like, not to the Gaussian noise alone. Gaussian noise of progressively increasing magnitude (see our answer to your question 3) is repeatedly applied to X_orig, and each noised structure is denoised using the base-model vector field plus a metaD-like bias repelling from all previously denoised structures:
noise X_orig by N(0,e1) → denoise repelling from {X_orig} → X_1 → noise X_orig by N(0,e2) → denoise repelling from {X_orig, X_1} → X_2 → etc.
We claim {X_orig, X_1, X_2, …} should resemble the sequence of states sampled by physics-based parallel-bias meta-dynamics. The bias independently stresses all native contacts, so we expect the weakest to break first - which is RipGuide's main goal.
What "accurate" unfolding pathways mean?
Thank you for pointing out this ambiguity.
It was meant to mean "a sequence of protein structures (X_1, X_2, X_3, … from our previous answer) that accurately mimics how the given protein would unfold under quasi-equilibrium heating from its original basin/conformation". We extended the discussion on this in Section 3.2.
- For a protein, an “accurate” unfolding “pathway” is an ensemble of physical unfolding trajectories under a specific disturbance, e.g. a temperature increase. In principle, this requires an exact specification of the disturbance, e.g. the exact schedule of how the temperature rises or how the salt concentration changes in time. First, we do not have dynamics when using AI-informed sampling (at least not without extra work to get it), so we can only model quasi-equilibrium processes. In our case, we aim to mimic quasi-equilibrium unfolding from a temperature increase. A quasi-equilibrium thermal unfolding started in a basin would spend some time exploring the initial basin, then “spill out” to a nearby basin (likely over the lowest barrier, but not necessarily), then the next basin, etc. This is exactly how meta-dynamics works if it is given a collective variable general enough to describe any feasible partially unfolded state. This is also how RipGuide is shown to work in Fig2c: correct basins are discovered and sampled in the correct (physically-expected) order before all basins are filled-up and random space-exploration begins. Specifically, a slow thermal disruption would approximately uniformly “stress” all native contacts in a protein, which is exactly what our parallel-bias does to all native contacts.
- Fig3(c,f) show the trajectories begin with thermal-like motions, then (Fig3(b,e)) local loop motions, then significant unfolding - the order we expect of a real unfolding. This is not proof that the process is fully physical, but it exhibits no clearly unphysical features.
Thank you for the response to my questions and concerns. While I can appreciate the motivation and the steps it took to arrive at ZipRipGuide, I am still struggling to see the necessity of the complexity/heuristics of this method over a simpler off the shelf sampler for diffusion models. For instance, the author's point about having to do a full denoising run with off the shelf samplers. This is not the case: one could noise a portion of the protein (that we want to stabilize), send this partially noised structure to an intermediate noise level (doesn't have to be the smallest or largest), and then conditionally sample a more stable version of only the portion we want to stabilize (similar to image inpainting). This main concern is in addition to other presentation concerns that make the paper (in its current form) difficult to justify accepting.
As a result, I will maintain my score.
Thank you for specifying your suggestion.
We believe the workflow you suggested in your last response is very close to what RipZipGuide does. If we understood correctly, there are 3 stages proposed:
noise a portion of the protein (that we want to stabilize),
We use RipGuide to localize which portion of the protein to noise. Simpler metrics such as pLDDT will not work as well (internal/external a-helices, disordered tails: see our answers to Q9tW). We just obtained results from an RipGuide ablation study (will add to the Supp soon) - disabling weak-region localization (i.e., removing the RipGuide step) breaks the stabilizing capability of ZipGuide, so RipGuide is an important step.
send this partially noised structure to an intermediate noise level (doesn't have to be the smallest or largest)
This is the 1st stage of ZipGuide: noise the structure to an intermediate level ( in the manuscript, Section 3.3.1)
conditionally sample a more stable version of only the portion we want to stabilize (similar to image inpainting).
This is close to the 2nd stage of ZipGuide (Section 3.3.2): denoise while applying the "stabilizing bias" to the identified weak region (which is mathematically equivalent to using DPS-sampling for conditional sampling). We are denoising the whole protein, but only the weak-region is allowed to mutate, so this is exactly "inpainting" of the weak-region part of the protein sequence. The whole protein needs to be denoised (from the intermediate noise level) because the geometrical arrangement/fold of the residues that do not mutate can still change residues geometrically close to the mutated region.
Response to Reviewer CEur Part 1
Thank you for highlighting potential improvements in the Methods and writing, and for suggesting possible simpler approaches to address the goal we had in the paper.
We have updated the Methods to avoid circular references and undefined terms. Additionally, just above the Discussion, we now describe how each stage of RipZipGuide contributes to the overall result. We plan to add more ablation studies closer to the Nov18 deadline.
We have merged Algo1 and the 1-2-3 list to avoid circular referencing and usage of undefined terms.
We added a brief note in the Introduction and more details at the end of Results on how each part of the workflow contributes and (in Supp Section C.1) on how hyperparameters were chosen. Qualitative arguments from the existing data for all existing steps are given below. We plan to conduct an ablation study to quantify the contribution of RipGuide in the workflow and report it closer to the Nov18 deadline.
Thank you for emphasizing this ambiguity.
This is partially addressed above. We added more discussion to address the particular choice of a stability proxy in the Introduction (H-bond counts, for its generality and locality) and the benefits of certain different parts of the proposed workflow (Intro and end of Results).
Fixed it, thank you for pointing out this omission. It means "Diffusion Posterior Sampling". We removed it from the title and added a non-acronym definition near where it is cited in Section 3.1, above eq.4.