- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Euterpe proposes an economical semi-online preference optimization framework to align pretrained ProteinMPNN toward protein developability properties while preserving inverse-folding performance. Instead of relying on costly folding-derived structural feedback as in MoMPNN, Euterpe uses signals from the frozen pretrained ProteinMPNN: a pair-level weight based on group-standardized reference sequence energy and token-level weights based on Top-K amino-acid support. On CATH 4.3, Euterpe matches MoMPNN closely in solubility, thermostability, designability proxies, and AAR, achieving up to a 6.9x per-epoch speedup in single-property settings and reducing to ~1.7x when evolutionary plausibility is included. The paper also provides global KL comparisons, component ablations, CASP15 TS45 transfer results, and an independent solubility evaluation via SoluProt.
Critical strengths
-
This paper has strong practical motivation and offers clear computational savings. Improving solubility or thermostability without degrading structural recovery is an important challenge in inverse-folding post-training. The detailed runtime and GFLOPs breakdown clearly shows that replacing repeated ESMFold evaluations with reference-model likelihood queries substantially reduces post-training overhead in single-property settings.
-
This paper includes informative empirical comparisons that help contextualize the reported gains. Beyond comparing against MoMPNN on CATH 4.3, the authors benchmark against -tuned DPO and explicit token-level KL regularization, analyze optimization dynamics, test zero-shot transfer on CASP15 TS45, and validate solubility with an independent predictor. These experiments support the practical feasibility of reference-guided preservation.
Critical weaknesses
-
The distinctive methodological contribution is not adequately isolated, and the pair-level module shows marginal utility. Preference-pair and token-wise DPO reweighting have clear precedents (e.g., WPO, TIS-DPO). Euterpe introduces nine heuristic hyperparameters () whose thresholds are retuned across tasks. Moreover, Figure 5(B) and Figure 8 show that token-level reweighting accounts for nearly all preservation gains, while removing pair-level reweighting or setting barely affects designability. At the token level, combining Top-K indicators from and into a shared weight ignores that and are evaluated under different decoding orders () and prefixes. The paper lacks controlled baselines using continuous reference probabilities, shuffled token weights, or simply plugging reference energy directly into MoMPNN (Eq. 4) in place of TM-score.
-
The central proxy assumption and broader generalization are insufficiently supported. High pretrained likelihood does not guarantee recovery of the target fold, especially on novel or out-of-distribution backbones, and this correlation is never quantified directly. Empirically, the evaluation omits the de novo and binder design settings studied in MoMPNN; on CASP15 TS45 (Table 7), Euterpe lags behind MoMPNN in pLDDT and RMSD < 3Å. Furthermore, thermostability is only evaluated by the training predictor (TemBERTure), Table 4 reports variance across five inference resamplings rather than independent training seeds, and the 6.9x speedup stems entirely from dropping ESMFold (shrinking to ~1.7x when EP is included in Table 2).
Questions for authors
-
Under the same training protocol, how does Euterpe compare with simpler alternatives to the proposed reweighting scheme, such as continuous reference-probability weighting, separately weighting the winner and loser branches under their own decoding contexts, or shuffled token weights with matched overall strength? It would also be helpful to compare against directly using the reference energy in MoMPNN’s multi-objective margin instead of TM-score.
-
How reliable is the frozen ProteinMPNN energy as a proxy for structural quality, especially for low-likelihood sequences and unfamiliar or OOD backbones? Could the authors report its correlation with ESMFold TM-score and RMSD across different energy ranges and provide examples where the proxy fails?
-
Table 2 suggests that most of the reported 6.9× speedup comes from removing ESMFold, while the advantage is closer to 1.7× when EP is included. Under a matched compute budget and comparable target-property performance, how does Euterpe compare with simpler no-folding baselines such as tuned or early-stopped DPO+KL on the structure–property trade-off?
-
Could the authors evaluate Euterpe on de novo backbones, as done in MoMPNN, and validate thermostability with an independent predictor? It would also be useful to report variation across independent training runs rather than only inference resampling, and to explain the drop in pLDDT and RMSD<3Å performance on CASP15 TS45 shown in Table 7.
2 comments
Response to Reviewer URrb (2/2)
Reliability of Pretrained Reference Energy as a Structural Proxy
Answer: We would like to clarify that Euterpe does not assume reference likelihood is equivalent to structural designability, nor that all structurally compatible sequences must receive high reference support. Our objective is to improve developability while preserving the designability and AAR already achieved by pretrained ProteinMPNN. The frozen IF model therefore serves as a preservation prior, rather than a predictor of structural quality.
Importantly, Euterpe employs soft reweighting rather than hard filtering, while semi-online rollouts continue exploring under the current policy. Thus, low-reference-support but structurally compatible sequences are not excluded, and their existence does not contradict our objective.
While correlations between reference energy and folding metrics could further characterize the prior, our method does not require reference likelihood to be a calibrated structural-quality predictor. Its effectiveness is instead evaluated directly through the preservation of designability and AAR during developability alignment.
Computational Efficiency and Simpler DPO/KL Baselines
Answer: We agree that the primary speedup comes from eliminating repeated ESMFold evaluations. This is precisely the computational bottleneck Euterpe aims to eliminate. Table 2 transparently reports the full cost breakdown, including the 6.9× speedup for single-property alignment and the approximately 1.7× speedup when EP evaluation dominates the computational cost. Our contribution is to achieve comparable property–designability trade-offs without expensive folding-derived feedback, rather than to accelerate components unrelated to structural preservation.
Besides, we have already investigated simpler alternatives in Fig. 4, including β-tuned DPO and DPO with different KL regularization strengths (equivalent to MoDPO+KL in the single-objective setting). At comparable solubility, Euterpe achieves better TM-score and RMSD-based preservation while maintaining comparable pLDDT, demonstrating a more favorable structure–property trade-off. Furthermore, Fig. 5(A) shows that even when property-only DPO surpasses Euterpe's final solubility at an early epoch, its structural preservation remains inferior. These results support the benefits of our reference-guided preservation beyond conventional KL regularization or early stopping.
CASP15 generalization
Answer: We acknowledge that Euterpe achieves lower pLDDT and RMSD < 3Å success rates than MoMPNN on CASP15 TS45. However, the comparison is mixed across metrics. Euterpe achieves higher TM-score (0.5904 vs. 0.5875), RMSD < 2Å success rate (0.3111 vs. 0.2815), and AAR (0.3591 vs. 0.3578), with nearly identical solubility (0.8537 vs. 0.8555). Euterpe also achieves higher solubility on 29 of the 45 targets.
Importantly, these results are obtained by directly transferring the same checkpoints and hyperparameters from CATH 4.3 to CASP15 TS45, without additional training or tuning. We do not claim uniform superiority across every metric, rather, these results support a comparable overall property–designability trade-off without folding-derived feedback.
Additional Evaluation and Limitations
Answer: Our study focuses on preserving pretrained IF capabilities during developability alignment, with extensive evaluation on held-out CATH 4.3 structures and zero-shot transfer to CASP15 TS45 without additional training or tuning. We follow the same evaluation protocol as MoMPNN for fair comparison and further validate solubility using the independent SoluProt predictor. These results support our conclusions within the intended scope. We will further clarify the distinction between inference-time resampling variability and independent training-seed variability, explicitly state the current evaluation scope, and discuss de novo/binder design and independent thermostability validation as future research directions in the revised manuscript.
We hope our responses have addressed your concerns, thank you!
Response to Reviewer URrb (1/2)
Thanks for your careful reviews!
Answer: We agree that token-level reweighting contributes most of the preservation gains. However, pair-level reweighting provides complementary sequence-level information by capturing reference-ranking conflicts and overall winner support, which cannot be fully characterized by local Top-K indicators. As shown in Fig. 5(B), combining both components achieves better preservation than token-level reweighting alone, demonstrating the benefit of jointly leveraging sequence-level and residue-level reference information.
Answer: We would like to clarify that Euterpe explicitly preserves the original decoding order sampled for each sequence during rollout. The winner and loser are evaluated separately under their respective decoding orders and autoregressive prefixes, as described in Section 4.3 and Eq. 8.
The resulting indicators characterize whether each residue is supported by the frozen reference model under its own generation context. They are subsequently combined to determine the relative contribution of the corresponding residue-wise preference signal. Therefore, our formulation does not assume that the winner and loser are evaluated under identical conditional distributions, nor does it directly compare their conditional probabilities as though they shared the same prefix. Instead, it combines their respective local support indicators to regulate preference optimization.
Answer: Euterpe's contribution goes beyond specific reweighting formulations: it demonstrates that pretrained IF knowledge can replace costly folding-derived feedback for preserving existing capabilities during developability alignment. We have already compared against property-only DPO, β-tuned DPO, explicit KL regularization, and component-wise ablations.
The suggested alternatives represent different possible design choices rather than established baselines with uniquely defined formulations. Our contribution goes beyond optimizing a particular weighting function, and we have already provided comparisons against property-only DPO, β-tuned DPO, explicit KL regularization, and component-wise ablations. While these alternatives may provide additional insights, our central contribution is to demonstrate effective preservation of pretrained IF capabilities without costly folding-derived feedback, rather than to establish the optimal weighting formulation among all possible variants.
Answer: We acknowledge that Euterpe introduces multiple hyperparameters through its two-level reweighting scheme. However, a larger number of hyperparameters does not necessarily imply a need for fine-grained tuning. Our sensitivity analyses in Appendix C.1 (Figs.~6--8) show generally stable performance across the tested configurations, with parameter changes primarily controlling the trade-off between developability alignment and preservation of pretrained IF performance.
For token-level Top-K, TM-score, pLDDT, and AAR remain within 0.729--0.745, 77.57--78.33, and 0.369--0.384, respectively, across all nine configurations. Lower solubility occurs mainly under the more conservative Kw=1 configurations, while the remaining six configurations achieve solubility scores of 0.870--0.926. For pair-level thresholds, solubility, TM-score, pLDDT, and AAR vary only within 0.885--0.898, 0.736--0.742, 77.84--78.17, and 0.374--0.378, respectively. The attenuation-coefficient analysis likewise shows stable structural preservation.
These results indicate that effective preservation is achieved across multiple tested configurations, rather than depending on a narrowly tuned hyperparameter combination. Moreover, most parameters are shared between the two single-property tasks in Table~5, and the same CATH checkpoints and hyperparameter configurations are directly transferred to CASP15 without additional tuning, further supporting their cross-benchmark applicability.