- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Energy-Guided Test-Time Refinement (E-TTR), a framework for improving full-atom peptide binder design through iterative refinement at inference time. Instead of relying solely on independent sampling, the method uses energy feedback to select promising candidates and refine unfavorable residues. The approach shows improvements in Rosetta binding energy compared with several existing test-time scaling strategies. Additional ablation studies investigate the contributions of the proposed components and different refinement settings.
Critical strengths
The paper is well organized, clearly written, and easy to follow, with effective figures illustrating the overall framework. The proposed approach is intuitive and well motivated, offering a reasonable strategy for improving peptide binder design through energy-guided refinement. The experimental evaluation is relatively comprehensive, covering multiple benchmarks and representative baselines, with ablation studies supporting the effectiveness of the proposed method.
Critical weaknesses
The main concern is that Rosetta energy is used both to guide refinement and to evaluate the generated designs. The independent-seed rescoring in Appendix E helps, but it is still Rosetta scoring Rosetta-optimized designs. Some non-Rosetta evaluation is needed before the dG improvements can be read as better binding. Second, while the candidate budget is matched across methods, E-TTR's refinement rounds are strictly sequential — each round must wait for Rosetta scoring of the previous one — whereas BoN sampling is fully parallel, and no wall-clock or compute measurement is reported anywhere in the paper, leaving the practical efficiency unclear. Third, I accept the authors' point that RMSD and AAR measure recovery of the native peptide rather than binding, but no measure of interface quality is reported in their place, so the consistent degradation of both metrics is left without evidence that the lower dG corresponds to physically plausible binding conformations. Finally, the paper reports no seed replication, variance, or confidence intervals, which makes the thinner margins hard to assess.
Questions for authors
- The proposed method uses Rosetta energy for both guiding refinement and evaluating the generated peptides. Could the authors provide an independent evaluation to verify that the improvements in dG reflect better binding quality rather than optimization toward the Rosetta scoring function?
- Could the authors report the actual inference time and computational cost of each method? Does E-TTR retain its advantage under a matched wall-clock time budget?
- Table 1 shows that E-TTR improves binding energy while yielding worse RMSD and AAR, particularly on PepBDB, where RMSD increases from 11.34 Å to 15.47 Å. Could the authors provide additional evaluations of binding interface quality to clarify whether the lower dG corresponds to physically plausible binding conformations?
- The main results do not report variability across independent runs or confidence intervals. In particular, the improvement in median dG on PepMerge is only 1.6% over BoN. Could the authors provide results across multiple random seeds or paired confidence intervals to assess the robustness of these improvements?
5 comments
Thank you for the detailed response and additional experiments. The new results have addressed most of my concerns and significantly strengthened the paper.
I have one minor clarification regarding Q4: Do the five random seeds correspond to independent runs of the entire generation and refinement process, or only to repeated Rosetta rescoring of a fixed candidate pool? This would help clarify the robustness of the reported improvements.
Thank you for your positive feedback and the follow-up question. The results reported in A4 correspond to repeated final Rosetta evaluations, with generation and refinement held fixed. Given the time required for full multi-seed generation and refinement, we were only able to complete the repeated final evaluations before our initial response.
We are now conducting additional independent runs of generation, refinement, and evaluation, and expect to report the results within 1–2 days to further assess robustness across the entire pipeline.
If our responses and additional experiments address your concerns, we would be grateful if you could consider increasing your score accordingly.
Response to Reviewer tWft
Thank you for the helpful comments!
W1 & Q1: Is improved binding quality supported by an evaluation independent of Rosetta?
A1: Our additional DeepPpIScore and MM/GBSA evaluations assess binding quality beyond Rosetta. Please see Section 2 of our general response for details.
W2 & Q2: Does E-TTR retain its advantage under a matched wall-clock time budget?
A2: We further compare all methods under matched time and NFE budgets at multiple cutoffs. Please see Section 3 of our general response and Figure 8 of the revised manuscript for the results.
W3 & Q3: Does lower dG indicate physically plausible binding despite worse RMSD and AAR?
A3: We evaluate the interface quality of all six methods on the three datasets using two metrics:
For each method and pocket, we compute both metrics on the candidate with the lowest dG and report medians across pockets with valid measurements.
Interface contacts. E-TTR achieves the highest CMS on PepBench and PepBDB, exceeding the strongest baseline by 6.84 and 21.83 Ų, respectively, and ranks second on PepMerge. These results indicate that its low-dG candidates also form extensive close contacts at the interface.
Area-normalized interface energy. E-TTR achieves the lowest Energy/area on all three datasets, outperforming the strongest baseline by 0.116, 0.065, and 0.105 on PepBench, PepBDB, and PepMerge, respectively. Its energy advantage therefore remains after normalization by buried surface area.
Together, these results support the plausibility of E-TTR's binding interfaces, showing that reduced native recovery can coexist with favorable interface contacts and area-normalized energy.
[1] Design of protein-binding proteins from the target structure alone. Nature 2022.
[2] InterfaceAnalyzer (https://docs.rosettacommons.org/docs/latest/application_documentation/analysis/interface-analyzer). Rosetta documentation.
W4 & Q4: Are the reported gains statistically robust across independent runs?
A4: We repeat the final evaluation using five random seeds and report the mean ± standard deviation.
We can see that E-TTR achieves the lowest median dG on all three datasets and the lowest mean dG on PepBench and PepMerge, ranking second on PepBDB. These results support the stability of E-TTR's energy improvements across evaluation seeds.