- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents EvoTilt-MPNN, a protein binder design method aimed at target sequence variation. Unlike the conventional practice of designing sequences against a single fixed wild-type target, the method describes the potential mutations at each target-interface residue as a per-residue amino-acid probability distribution π, and introduces a KL-divergence-based distributional tilt during ProteinMPNN's autoregressive decoding, so that the tilted inverse-folding posterior on the target side moves closer to π. This biases the generated binder toward tolerating these potential substitutions without explicitly enumerating mutant structures. To contain the extra cost, the authors design a cache-based leave-one-out decoding scheme that keeps the runtime of the tilted procedure on the same order as vanilla ProteinMPNN.
Experiments span multiple target families. The results show that the method consistently reduces the distributional divergence at the target interface, and that this effect partially transfers to inverse-folding models not used for optimization. On the mutant co-folding evaluation for a subset of these families, mild tilting yields a small, target-heterogeneous, worst-tail-concentrated improvement in structural confidence, whereas stronger tilting does not sustain the gain and causes some metrics to fall back below baseline. The paper demonstrates the feasibility of incorporating a mutation prior directly into sequence generation, but the improvement in actual binding robustness remains insufficiently validated.
Critical strengths
The problem studied is of clear biological significance. Compared with designing sequences against a single target structure, modeling potential target mutations as a per-residue probability distribution offers a general route to variation-aware binder design that does not require explicit enumeration of mutant structures and can be fed priors from DMS, phylogenetic frequencies, or escape predictors. Using the target-side inverse-folding posterior for distributional control at the generation stage carries some methodological novelty.
The computational optimization is sound and well validated. By caching the leave-one-out decoding states, the authors substantially reduce the overhead of distributional tilting, keeping its runtime on the same order as vanilla ProteinMPNN; more importantly, multiple exact implementations produce identical sequences, which gives the "exactness" claim hard support rather than resting on a speedup figure alone.
The authors test distribution matching across multiple families and verify cross-model transfer using different ProteinMPNN checkpoints and ESM-IF1. By decomposing the changes in WT probability versus non-WT preferences, they provide evidence that the KL improvement under mild tilting stems mainly from preference reallocation among non-wild-type residues rather than simply inflating the predicted probability of the wild-type residue, effectively ruling out the trivial explanation that tilting merely makes the model more confident in the WT.
Critical weaknesses
The main weakness is that the core improvement in binder robustness lacks sufficiently strong evidence. The distribution-matching metrics are stable, but in the primary structural evaluation the gain in worst-decile interface iPTM is small and positive in only about half of the families; as the paper itself notes, most worst-decile readouts improve under mild tilting yet largely drift back toward baseline under stronger tilting. With such a narrow effective window and no reported cross-family statistical significance (e.g., a paired test or bootstrap CIs), it is hard to judge whether this small, heterogeneous improvement is reliable—especially since the number of interface positions hit by the mutation panel varies across designs, further affecting worst-tail stability.
The method is also not validated against adequate competitive baselines. The experiments mainly compare tilt strengths against vanilla ProteinMPNN, without a matched-budget comparison to post-hoc reranking under the same KL objective. Since decoding-time tilting can reach sequences outside the vanilla sampling pool while reranking is confined to existing candidates, the two explore different spaces; absent such a comparison, it is unclear whether tilting at decoding time is actually superior to simple post-generation filtering.
A clear gap remains between distribution matching and genuine binding robustness. All experiments use a single ESM-2-inferred prior, without relating it to real mutation frequencies, measured mutational tolerance, or binder escape; the paper also concedes that the interface divergence weights all positions equally, whereas binding is decided by a subset π knows nothing about—so the weak effect may stem from a prior misaligned with binding-sensitive positions rather than from the method. The structural evaluation further relies mainly on Boltz-2 with BoltzGen-generated backbones, which the authors admit are not fully independent, and the held-out transfer verifies only the distributional proxy, not structure or function. The results thus more directly support a small gain in structure-prediction confidence than in actual binding robustness.
Finally, some controls and approximations are under-validated: the anti-π arm is close to uniform, limiting its power as an adversarial control; the prior-specificity result is exploratory with CIs including zero; and the greedy per-position approximation to the joint objective of Equation 2 is never quantified, leaving open whether it partly explains the "strong distribution matching but weak structural gains" pattern.
Questions for authors
- The worst-decile interface iPTM improves in only about half of the families. Could you report a paired statistical test across families, bootstrap confidence intervals, and leave-one-family-out sensitivity analysis? Also, how many target-side interface positions are covered by the mutation panel per design, and is this coverage sufficient for a reliable worst-decile estimate?
- Under matched candidate counts and compute budgets, how does EvoTilt-MPNN compare with generating sequences using the original model and reranking them using the same KL-based objective? Does decoding-time tilting provide worst-tail robustness gains beyond those achievable through post-hoc reranking, particularly as the candidate pool increases?
- Since the backbones are generated by BoltzGen and structural robustness is mainly evaluated using Boltz-2, could you validate the results using an independent structure predictor or physics-based interface assessment? Does the worst-tail improvement persist under this independent evaluation?
- The method relies on ESM-2 marginals as mutation priors. For targets with available deep mutational scanning (DMS) data, could you construct an alternative prior from experimentally measured mutation preferences and compare the resulting structural robustness? This would help determine whether the limited structural improvement is primarily due to the quality of the mutation prior or the tilting method itself.
2 comments
4. Does the method generalize beyond the ESM prior, including to DMS-derived priors? We agree that a functional DMS-derived prior would be the most direct test of whether limited structural improvement is caused by prior quality. Constructing and biologically validating such a prior is an important but distinct task from the problem addressed here—(designing against a supplied prior) We did not complete a DMS-prior structural campaign but have added it as an obvious next step and future work. We therefore cannot separate those two explanations experimentally. We did test the narrower question of whether the distributional mechanism is specific to ESM by repeating inverse design for all 15 primary families with an independently constructed, untruncated MSA-frequency prior. Target-side KL decreases in 15/15 families at beta=10 and beta=20. Reciprocal cross-scoring favors the matched prior in both directions: the matched-minus-crossed gain is +0.00946 at beta=10 (95% CI [+0.00683,+0.01268]) and +0.01970 at beta=20 ([+0.01412,+0.02640]; Section 5.2; Appendix E). This establishes posterior-level transfer across two prior sources, not structural generality of the MSA campaign or validation of homolog frequency as functional mutational tolerance.
Additional concerns: anti-prior control and the stepwise approximation We agree that the anti-pi panel is close to uniform and therefore only mildly adversarial; the revision no longer treats it as strong falsification evidence. Structural prior-specificity is also reported as exploratory: at beta=10, the on-interface pi-versus-uniform contrast is +0.0104 (95% CI [-0.0037,+0.0245]), while the off-interface contrast is near zero. Stronger evidence that the posterior response is prior-directed comes from the reciprocal MSA/ESM cross-scoring above and from the conditional-mutant objective, for which the pi-weighted reduction exceeds the uniform-weighted reduction in all 21 families.
Finally, caching is exact for the defined stepwise tilted proposal: independent exact implementations produce identical sequences. The greedy per-position proposal remains an approximation to the global joint tilt in Equation 2. Evaluating the terminal joint cost for a completed sequence is tractable and shows consistent KL reduction, but exact sampling from the global tilt would require marginalizing over exponentially many sequence completions. We did not quantify the distance between these two distributions and therefore cannot exclude this approximation as one contributor to the distributional--structural gap. The revised Methods and Limitations make this distinction explicit. However, as a small toy test case, we separated cache exactness from the greedy approximation for a single A1M case: exact enumeration over combinations at the eight most affected binder positions gave and TV , while full-sequence importance sampling over all 93 binder positions gave , TV , and showed that the stepwise proposal retained 93% of the ideal objective reduction. This single case study suggests that the approximation is modest in the tested setting, but does not establish equivalence across decoding orders, backbones, or families..
We thank the reviewer again: these analyses sharpen both the evidence for mild-tilt distributional and structural effects. We hope the new analysis help address the most pressing concerns.
We thank Reviewer jBSD for the careful and technically engaged review. We appreciate their recognition of the significance of variation-aware binder design, the methodological novelty of using the target-side posterior for distributional control, and the soundness and exact implementation agreement of the cached decoding optimization. Their questions motivated a substantially stronger evaluation and clearer statement of the method's scope.
1. Is the structural improvement statistically reliable, and is panel coverage sufficient? We agree that the original paper did not provide sufficient uncertainty analysis. The primary worst-decile interface-iPTM effect is +0.0176 (family-bootstrap 95% CI [+0.0001,+0.0374]; paired t-test p=0.098; 9/15 families favorable). It is therefore not resolved by the paired test at the available family-level power. However, the mean remains favorable under every leave-one-family-out omission. The response also persists as the tail broadens: +0.0106 at eta=0.25, +0.0051 at eta=0.50, and +0.0044 for the full on-interface mean. Complex PDE moves concordantly (-0.0252; 95% CI [-0.0495,-0.0006]; paired p=0.072; 10/15 favorable), and 12/13 worst-tail confidence readouts are favorable at beta=10 (Section 5.4; Appendices B and F).
We also quantified the requested panel coverage. The pi panel intersects a median of 10 distinct target-interface positions per design (IQR 7--12; range 2--30), or 25.5% of the modeled target interface. Because CVaR_0.10 uses max(1,floor(0.10 n_on)) mutations, its tail contains one mutation for 94.9% of primary design-condition rows. We now describe this endpoint as a conservative worst-observed on-interface stress test rather than a precisely resolved within-design decile; the broader-tail estimates above provide the corresponding higher-coverage sensitivities (Appendix A.6).
2. Does decoding-time tilting improve over post-hoc reranking as the candidate pool grows? We computed terminal KL for all 7,249 eligible vanilla candidates across the 15 families and selected the same 40-design output count. Pure-KL selection reduced terminal KL, but WT interface iPTM fell from 0.8283 to 0.6564 and mutant CVaR_0.10 from 0.6300 to 0.4920 (Delta=-0.1381; 95% CI [-0.1668,-0.1100]). Confidence-selected EvoTilt retained WT iPTM at 0.8302 and achieved mutant CVaR_0.10 of 0.6476, exceeding vanilla pure-KL selection by +0.1557 ([+0.1256,+0.1843]; favorable in 15/15 families).
We also evaluated the full model-free confidence--KL curve and structurally tested an intermediate M=60 policy, which selects the lowest-KL 40 designs among the 60 highest-confidence candidates. This compromise did not improve absolute CVaR for either vanilla or EvoTilt candidates. Thus the tested post-hoc selectors do not reproduce EvoTilt's confidence--robustness trade-off as the candidate pool increases (Section 5.6, Table 3; Appendix G). Because the selected cohorts differ, this is a comparison of complete generation-and-selection policies rather than a same-sequence causal decoder ablation.
3. Does the response persist under an alternative mutation-effect evaluation? We evaluated the same frozen 15-family cohort with FoldX and StaB-ddG. For the pi-arm, design-interface worst tail, FoldX changed favorably by -0.330 (95% CI [-0.552,-0.141]; 12/15 families; multiplicity-adjusted p=0.0061). StaB-ddG changed by -0.071 ([-0.133,-0.011]; 10/15; adjusted p=0.0891), providing directional but not multiplicity-adjusted evidence. The FoldX direction persisted after adding each design's WT interaction energy (-1.314; [-2.168,-0.580]; 12/15 families), arguing against an apparent mutant gain caused only by worse WT complexes (Section 5.7, Table 4; Appendix H). These alternative mutation-effect scores broaden the endpoint beyond Boltz-2, but they remain computational and start from Boltz-derived structures; we therefore do not present them as a fully independent structure-generation test or as experimental binding validation.