acceptodds
Desk rejected at ICLR 2027

ProteoCraft: Protein Design as Proposal–Verification Search

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Protein–protein interactions underpin nearly all biological processes. Yet, designing proteins with desired interactions remains challenging due to the structural complexity governing molecular recognition. This study views protein design as a search optimization problem over a combinatorial sequence space under expensive structural evaluation. Toward this goal, we introduce a novel tree-based search framework, ProteoCraft, that employs approximate structural gradients from diffusion-based structure predictors as effective proposal policies to specify where the search should explore. Complete no-gradient inference of folding models separately assigns values to the resulting sequences, determining what is good. ProteoCraft allocates this verification budget through a coarse-to-fine paradigm: multi-site edits first explore distinct sequence basins, then verified promising states undergo strict single-site refinement. Across fourteen de novo binder-design settings spanning eight target proteins, five immune-relevant TCR-facing targets, and four localized CDR redesign tasks, ProteoCraft improves design quality and high-confidence yield. These improvements persist under independent AF2-IG and Rosetta-based interface evaluation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein Design, Hallucination, MCTS.

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ftScOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Proteocraft proposes mutation though gradient based mutataion proposal and MCTS. Compute gradients, identify possible mutations, generate, run cofolding model, evaluate the confidence and then use mcts to choose which sequence to expand then repeat. It first represents a.a as continuous probabilities then calculates the gradient signal using distogram, seq prior and one-step diffusion. core novelty is from the one-step diffusion gradient,

Critical strengths
  1. Performance seems good, however, i leave a question regarding how fair performance comparison is
  2. Wet-lab results seem promising this supports that the method works
  3. One-step diffusion contribution seems clear
Critical weaknesses
  1. Novelty is questioned. What is already established: Continuous amino acid logits and gradient-based optimization, using protein language models as sequence priors, distogram-based sequence optimization, evaluating mutated sequences with structure predictors. UCB/PUCT-based tree search, iterative sequence refinement. All above already exist. One-step diffusion signal is not novel and coarse-to-fine is just an algorithmic choice.
  2. I question the headline claims that ProteoCraft significantly outperforms existing binder-design methods. Different computational budgets. Table 1 compares 210 candidates per method. But ProteoCraft performs substantial optimization before selecting its final candidates: 70 Stage-A search iterations, evaluating up to 8 candidates per iteration, 40 additional Stage-B refinement iterations, Multiple initial sequences and repeated full ESMFold2 evaluations.Meanwhile, competing methods use their native generation procedures. You must report BoN(for all design methods) or TTA methods(complexa) in each paper along with GPU-h or budget analysis. Otherwise, it's hard to accept this claim. Figure 1's quality-versus-runtime comparison is therefore not a standardized comparison of the cost to produce one final high-quality binder. The authors should report GPU-hours per design passing the same independent evaluation threshold for every method
Questions for authors
  1. paper does not provide enough information about the diffusion noise schedule to determine exactly how many denoising steps occur before τ = 2. Please elaborate regarding the one-step diffusion gradient.
  2. The method assumes: Gradients from approximate structural objectives provide useful directions for discovering sequences with better full-model confidence. But the paper does not directly demonstrate how reliably that assumption holds. For example, suppose the gradient recommends 100 amino acid substitutions. How many actually improve pair iPTM after complete structure prediction? And how does this compare with random mutations, ESMC-based mutations, or distogram-only gradients? That experiment would tell us whether their gradient proposal mechanism is genuinely informative. Without it, we cannot easily distinguish two explanations for their success:
  3. The main benchmark contains 14 target–hotspot settings, but only eight unique proteins. In particular nine of the fourteen settings come from just three proteins. This can inflate the apparent diversity of the benchmark.The authors also do not provide convincing uncertainty estimates across independent design runs or target proteins. I would prefer evaluation on a substantially larger and more diverse target set, with uncertainty reported at the protein level. For more targets, refer to ProtDBench targets and please add Boltzgen hard targets(in examples). include protein binders and nanobodies.
  4. More cofolding model confidence results, use opendde for instance and protenix-v2.
  5. Where are the TTA methods from complexa paper? Beam/MCTS/BoN/Feynman-kac sampling? Why exclude these in the performance? The authors state that the internal variants use matched candidate/verifier budgets. This provides evidence that MCTS improves the authors' own proposal-based optimization pipeline. However, it still does not establish whether MCTS is better than beam search or a conventional evolutionary algorithm using the same proposals.
  6. I would require a compute-matched benchmark: Fix the same target structures, hotspots, and binder lengths. Give every method the same GPU-hour budget. Permit each method to generate and filter candidates using its own pipeline. Evaluate the resulting candidates using independent structure predictors. Compare the number of high-confidence designs obtained per GPU-hour, along with structural diversity. Report results separately for each protein, with confidence intervals across independent runs. If this is too much, refer and use ProtDBench. Please add TTA methods as i questioned in 5.
  7. Report diversity of generated binders using foldseek or other tools. report how diverse, threshold-passing binders are made. Efficiency should focus on that.
  8. Please compare performance with RFOptimization.
13 comments
AuthorsOpenReview ↗

Reply to Reviewer ftSc (Sec I)

Thank you for the detailed assessment and for recognizing the promising performance, experimental binding results, and contribution of one-step diffusion guidance. Your questions motivated additional mutation-level controls, a beam-search comparison, and a broader ProtDBench evaluation under a measured A100 GPU budget. Below, we clarify the methodological contribution and report the additional evidence on proposal quality, computational efficiency, and structural diversity.

Q1: Exact diffusion schedule and the one-step gradient

In our ProteoCraft implementation, setting the truncation noise level to τ=2\tau=2 with the 50-point clipped schedule produces 22 reverse-sampling updates before the terminal differentiable denoiser call. This includes the update that reaches τ=2. All 22 updates run without gradient tracking. At every one-step gradient evaluation, the sampler starts from Gaussian noise and recomputes the trajectory using the current soft sequence conditioning. We then apply stop-gradient to the newly sampled XτX_\tau, evaluate the frozen denoiser once at τ=2, and differentiate the structural loss through this terminal call and the soft sequence conditioning with respect to the sequence logits. We will include the schedule construction and the distinction between forward sampling and backward differentiation in the method description.

Q2: Whether gradient proposals identify beneficial discrete mutations

Thanks for your question. Our existing No One-step Diffusion ablation retains the distogram and ESMC signals. Before tree search, adding the one-step diffusion signal increases mean pair-chain iPTM from 0.637 to 0.660 and the number of candidates above 0.7 from 85/210 to 118/210, supporting its contribution to proposal quality.

To directly assess individual substitutions, we additionally compared five proposal strategies: random mutations, ESMC-only, distogram-only, distogram+ESMC, and the full method. Each strategy was evaluated on 128 single-residue substitutions using complete structure prediction. We report both the mean change in pair-chain iPTM and the fraction of mutations whose improvement exceeds 0.01.

# Soft-update StepsRandomESMC-onlyDistogram-onlyDistogram+ESMCFull
513/12817/12815/12815/12816/128
1013/12811/12811/12818/12821/128
1513/12811/12815/12817/12822/128
2013/12810/12814/12818/12823/128

The full method produces 23/128 mutations with improvement above 0.01, compared with 13/128 for random mutations and 18/128 for distogram+ESMC. The improvement over random is 7.8 percentage points, with a paired 95% confidence interval of +0.8 to +16.4 percentage points. The difference from distogram+ESMC is 3.9 percentage points, with an interval of −7.8 to +14.8 percentage points.

# Soft-update StepsRandomESMC-onlyDistogram-onlyDistogram+ESMCFull
5+0.002096+0.002496+0.002629+0.003791+0.003592
10+0.002096+0.002508+0.001554+0.004132+0.004447
15+0.002096+0.002312+0.001851+0.003180+0.004244
20+0.002096+0.001905+0.001744+0.003458+0.004560

At the 20 soft updates used in Stage B, the mean changes across all substitutions are +0.002096 for random mutations, +0.001905 for ESMC-only, +0.001744 for distogram-only, +0.003458 for distogram+ESMC, and +0.004560 for the full method.


These results provide direct evidence that the full proposal mechanism enriches mutations with larger improvements compared with random sampling. Its incremental advantage over distogram+ESMC remains uncertain in this experiment. We will state this distinction alongside the existing end-to-end ablation.

Q3 / Q6 / W2: Broader target coverage and standardized quality–cost evaluation

We agree that the original benchmark contains 14 target–hotspot settings from 8 unique proteins, and that these settings should not be treated as 14 independent proteins. We will state both counts consistently.

Moreover, we agree that broader target coverage is valuable, and we have added the requested ProtDBench evaluation. However, assessing practical binder-design performance primarily through confidence-filter pass rates requires an additional assumption: that passing the chosen filter is sufficiently predictive of experimental binding across targets and design methods. The available evidence shows substantial limitations to this assumption.

AuthorsOpenReview ↗

Reply to Reviewer ftSc (Sec II)

(1) ProtDBench [A] itself provides direct evidence for this concern. Its analysis of experimentally annotated designs shows substantial dependence on the verifier, confidence metric, and target. It further shows that different verifiers recover different subsets of experimentally confirmed binders, and that recall decreases rapidly as more verifiers are required to agree. Consequently, using a common confidence threshold standardizes the computational evaluation, while the relationship between that threshold and experimental success still requires validation. In particular, a higher number of filter-passing designs per GPU-hour cannot, by itself, establish a higher number of experimentally functional binders per GPU-hour.

[A] ProtDBench: A Unified Benchmark of Protein Binder Design and Evaluation. ICML 2026

(2) Further evidence [B] analyzed 3,766 experimentally tested de novo binders across 15 targets. Although AF3 ipSAE_min was the strongest individual predictor, its predictive performance varied substantially across targets. Even at target-specific thresholds selected to maximize F1, the corresponding precision ranged from approximately 0.1 to 1.0, with limited positive counts contributing uncertainty for some targets. Combining confidence scores with physicochemical interface descriptors, including Rosetta δG/δSASA and shape complementarity, further improved prediction. These findings demonstrate that a confidence threshold does not establish a consistent experimental success probability across targets. They support evaluating design methods through complementary computational measures and prospective binding experiments.

[B] Predicting Experimental Success in De Novo Binder Design: A Meta-Analysis of 3,766 Experimentally Characterised Binders. 2025.09

(3) We also investigated this issue further through our retrospective analysis of 1,230 experimentally tested designs from the Anthropic dataset (https://huggingface.co/datasets/Anthropic/claude-protein-binder-design) [C]. Discrimination varied substantially by target: for ipSAE-based screening, the median AUC across predictors was approximately 0.481 for BHRF1 and 0.497 for Cas9, compared with 0.880 for Latent GDF-8. On the 12 targets with at least 5 experimental positives, selecting the top 10% of designs within each target by ipSAE yielded reported hit rates of 48.8%–61.9% across individual predictors. Thus, approximately 38%–51% of these highly ranked candidates still failed experimental binding assessment. Combining all ten predictors increased the reported hit rate to 63.1%, leaving approximately 37% experimental failures. Confidence ranking enriched hits relative to random selection, but substantial uncertainty remained even among the highest-ranked candidates. These results illustrate why confidence-filter pass rates require experimental calibration before they can be interpreted as functional binder yields.

[C] Autonomous de novo protein binder design with Claude. 2026.08

(4) Complementary evidence [D] from 106 experimentally determined nanobody–antigen complexes and 11,342 shuffled non-cognate pairings showed that AF3, Boltz-2, and Chai-1 frequently produced geometrically plausible complexes, while their ipTM scores had limited ability to distinguish cognate from non-cognate pairings. Increased sampling improved structural refinement but provided limited improvement in pairing discrimination. This finding is particularly relevant to our VHH setting, and further motivates experimental binding and specificity measurements when assessing design success.

[D] Structural Plausibility Without Binding Specificity. 2026.03

(5) Experimental binding affinity introduces a further distinction. The Adaptyv EGFR competition analysis [E] found no statistically significant correlations between measured affinity and the metrics used to rank designs, including ipTM and iPAE. The POISK antibody benchmark [F] similarly reported only a weak relationship between structural confidence and affinity across 5,207 records with usable affinity or potency measurements. These studies show why increases in computational confidence alone provide limited evidence about improvements in binding strength.

[E] Crowdsourced Protein Design: Lessons From the Adaptyv EGFR Binder Competition. 2025.04

[F] Can Folding Models Tell Binders from Bluffers? Evidence from POISK: The Patent-Derived Antibody Dataset. NeurIPS 2026


To summarize, a larger confidence-based target panel would not be treated as a substitute for prospective experimental validation or as direct evidence of improved binding success. We therefore consider prospective experimental validation essential when assessing whether a design method produces functional binders.

AuthorsOpenReview ↗

Reply to Reviewer ftSc (Sec III)

For PD-L1, 29 of 35 tested miniproteins showed measurable binding, yielding an 82.9% hit rate, the highest observed rate among the studies listed in our comparison. This compares with reported rates of 82.4% for PXDesign, 77.8% for BindCraft, 72.0% for Chai-2, and 10.0%–22.0% for the other listed methods. The best ProteoCraft binder achieved a fitted KDK_D of 0.879 nM, demonstrating subnanomolar affinity. Our IFNAR2 experiments additionally yielded 4 binding hits among 9 successfully expressed VHH candidates, with a best fitted KDK_D of 26.29 nM. Together, these prospective measurements demonstrate experimental binding across two targets and two binder formats, complementing our computational evaluations.

MethodHit rateHits / TestedBest K_d (nM)
Proteocraft82.90%29/350.879
PXDesign82.40%14/1721
BindCraft77.80%7/9615
Chai-272.00%18/250.982
Latent-X22.00%22/1000.27
AlphaProteo15.00%NA0.18
RFdiffusion12.60%NA1.6
BoltzGen10.00%2/2096

Additional evaluation on ProtDBench

Benchmark construction and implementation. To address the reviewer’s request for broader computational coverage and fair comparison with the same computational budget, we nevertheless evaluated ProteoCraft on all ten ProtDBench targets: BHRF1, PDL1, IR, TrkA, IL7RA, SC2RBD, VEGFA, H1, IL17A, and TNFα. To cover multiple binder lengths within the supplementary evaluation budget, we selected five fixed lengths (80, 90, 100, 110, and 120 residues) shared by all targets. For each target–length combination, ProteoCraft contributes four backbones, yielding 20 backbones per target and 200 backbones overall. We extracted baseline results for the corresponding target–length combinations from the official ProtDBench release. BindCraft contributes 291 backbones across these combinations, while each other method contributes 200.

We sampled 8 ProteinMPNN sequences per backbone and evaluated them using the shared independent AF2-IG protocol. This produces 1,600 evaluated sequences per method for the 200-backbone cohorts, and 2,328 for BindCraft. The table below reports sequence-level pass rates under the AF2-IG Easy and Strict filters. We first calculate the pass rate for each target–length combination, average the five lengths with equal weight within each target, and then average the ten targets with equal weight. This aggregation prevents targets or length settings with more evaluated samples from dominating the reported means.

Independent AF2-IG evaluation. ProteoCraft achieves mean Easy and Strict sequence pass rates of 54.50% and 27.06%, respectively. The strongest baseline results are BindCraft’s 33.88% Easy pass rate and PXDesign’s 11.94% Strict pass rate. Using the unrounded values, ProteoCraft improves these by 20.62% and 15.13%, respectively.

The improvement extends across most targets. ProteoCraft has the highest Easy pass rate on nine of ten targets and the highest Strict pass rate on nine targets. On H1, all methods have a Strict pass rate of zero. ProteoCraft’s strongest results include PDL1, with Easy/Strict pass rates of 91.88%/69.38%, BHRF1 at 76.88%/54.38%, IR at 67.50%/52.50%, and TrkA at 75.00%/49.38%.

Method (Easy / Strict)BHRF1PDL1IRTrkAIL7RASC2RBDVEGFAH1IL17ATNFαAvg.
ProteoCraft76.88 / 54.3891.88 / 69.3867.50 / 52.5075.00 / 49.3871.88 / 1.8856.88 / 30.6331.25 / 8.133.75 / 0.0026.25 / 1.2543.75 / 3.1354.50 / 27.06
BindCraft44.46 / 17.7470.42 / 42.0253.66 / 19.9647.96 / 25.4544.52 / 0.0017.67 / 3.5010.36 / 0.5030.46 / 0.0019.24 / 0.420.00 / 0.0033.88 / 10.96
PXDesign52.50 / 40.0048.13 / 36.2543.13 / 22.5033.75 / 16.2526.25 / 0.007.50 / 1.2518.75 / 3.1320.00 / 0.000.00 / 0.002.50 / 0.0025.25 / 11.94
BoltzDesign-16.25 / 2.5029.38 / 7.5011.88 / 0.0040.63 / 8.756.25 / 0.006.25 / 0.630.63 / 0.006.88 / 0.008.75 / 0.000.00 / 0.0011.69 / 1.94
BoltzGen13.75 / 13.1320.00 / 10.0015.00 / 6.887.50 / 0.002.50 / 0.001.25 / 0.006.88 / 0.0027.50 / 0.000.00 / 0.000.00 / 0.009.44 / 3.00
ODesign38.75 / 11.883.75 / 0.008.13 / 3.7510.00 / 0.0018.75 / 0.005.63 / 2.501.88 / 0.006.88 / 0.000.00 / 0.000.00 / 0.009.38 / 1.81
RFdiffusion-335.00 / 15.0010.00 / 5.0012.50 / 8.753.13 / 1.8811.25 / 0.000.63 / 0.002.50 / 0.001.88 / 0.000.63 / 0.000.00 / 0.007.75 / 3.06
Protpardelle-1c0.00 / 0.005.00 / 0.631.25 / 0.001.25 / 0.000.00 / 0.005.00 / 0.000.00 / 0.001.88 / 0.000.00 / 0.000.00 / 0.001.44 / 0.06
Reviewer_pknhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ProteoCraft designs protein binders by combining gradient-guided sequence changes with tree search. Approximate gradients suggest mutations, while complete structure predictions evaluate the resulting sequences. Computational benchmarks, independent structural evaluations, and binding experiments on two targets support the approach.

Critical strengths
  1. The ablations provide empirical support for the design choices. Comparisons with a matched Best-of-N baseline suggest that tree search and diffusion-based guidance contribute to the reported gains.

  2. The paper includes additional validation. AF2-IG and Rosetta evaluations provide evidence beyond the search-time predictor, while binding experiments on two targets offer initial experimental support.

Critical weaknesses
  1. The reliability of the truncated guidance remains insufficiently characterized. The gradient holds the intermediate structure fixed, ignoring how sequence changes would affect the preceding diffusion trajectory. The ablation and noise-level study imply its usefulness, but do not establish how reliably it identifies beneficial discrete mutations or how sensitive it is to the preceding sampling steps and schedule. This leaves uncertainty about its robustness across inference configurations.

  2. Experimental validation does not establish the overall success rate or an advantage over existing methods. The paper reports successful binders against two targets, but does not clearly report the total number of candidates tested, unsuccessful designs, or experimental baseline comparisons.

  3. The runtime comparison uses inconsistent definitions of an output candidate. Appendix C.3 counts qualifying intermediate search sequences as ProteoCraft outputs, while other methods are measured per final design or per design passing their own filters. These differences are disclosed, but prevent a direct comparison of computational efficiency. A common quality criterion and a consistent definition of output are needed to substantiate the reported trade-off between quality and cost.

Questions for authors

Please see weaknesses

2 comments
AuthorsOpenReview ↗

Reply to Reviewer pknh (Part I)

Thank you for identifying the questions about truncated-gradient reliability, experimental success rates, and runtime accounting. We have added direct tests of discrete mutation proposals, expanded prospective binding experiments, and a throughput comparison using a common acceptance criterion. We also clarify how the diffusion trajectory is recomputed at each gradient evaluation and describe the scope of the sensitivity evidence.


1. Reliability and Sensitivity of Truncated Diffusion Guidance

Thanks for your valuable question! We clarify that XτX_\tau is recomputed at every one-step gradient evaluation by running reverse diffusion from Gaussian noise under the current soft sequence conditioning. The stop-gradient is applied after this forward trajectory has been recomputed. The truncated derivative therefore omits backpropagation through the preceding sampling updates, while the forward computation uses the current conditioning throughout.

We additionally evaluated 128 single-residue substitutions per proposal strategy using complete structure prediction. At 20 soft updates, the full method achieved the largest observed mean change in pair-chain iPTM, +0.004560, compared with +0.002096 for random mutations and +0.003458 for distogram+ESMC. Improvements exceeding 0.01 occurred for 23/128, 13/128, and 18/128 substitutions, respectively. The full method’s improvement over random was 7.8 percentage points, with a paired 95% confidence interval of +0.8 to +16.4 percentage points. Its incremental difference from distogram+ESMC remained uncertain, with an interval spanning zero.

# Soft-update StepsRandomESMC-onlyDistogram-onlyDistogram+ESMCFull
513/12817/12815/12815/12816/128
1013/12811/12811/12818/12821/128
1513/12811/12815/12817/12822/128
2013/12810/12814/12818/12823/128

Table 13 evaluates truncation levels τ∈{1,2,4}\tau\in\{1,2,4\}, yielding mean pair-chain iPTM values of 0.682, 0.777, and 0.730 across three matched targets. These results characterize cutoff sensitivity under the tested sampler.

2. Experimental Hit Rates and Scope of Validation

Thanks for raising the question. We have expanded the experimental reporting to include both binding hits and unsuccessful candidates. For PD-L1, 29 of 35 tested miniproteins showed measurable binding, corresponding to an 82.9% hit rate, with six candidates showing no measurable binding under the assay conditions. The best-fitted KDK_D was 0.879 nM.

Reported hit rates in the literature (context only; not a controlled comparison)

MethodHit rateHits / TestedBest K_d (nM)
Proteocraft82.90%29/350.879
PXDesign82.40%14/1721
BindCraft77.80%7/9615
Chai-272.00%18/250.982
Latent-X22.00%22/1000.27
AlphaProteo15.00%NA0.18
RFdiffusion12.60%NA1.6
BoltzGen10.00%2/2096

For IFNAR2, four of nine successfully expressed VHH candidates and five of 27 successfully expressed miniproteins showed binding, corresponding to expression-conditioned hit rates of 44.4% and 18.5%, with a best-fitted KDK_D of 26.29 nM.

These experiments establish functional binding among the tested ProteoCraft candidates. The literature comparison provides context, while differences in selection and assay protocols limit conclusions about relative experimental performance. A direct experimental comparison under shared conditions remains a limitation.

3. Consistent Output Definitions and End-to-End Computational Efficiency

We agree that the original runtime comparison used different eligibility rules for counting outputs. We have added a ProtDBench comparison using a shared output definition: a complete backbone qualifies when at least one of its eight ProteinMPNN sequences passes the common independent AF2-IG Easy filter. Complete candidates discovered throughout the ProteoCraft search are eligible under this same rule, allowing a search run to contribute multiple outputs.

The cost accounting includes search or generation, sequence design, and independent evaluation. ProteoCraft’s estimated mean yield is 247.9 passing backbones per 24 A100 GPU-hours, compared with 533.3 for PXDesign, 318.5 for BoltzGen, 293.6 for ODesign, 269.0 for RFdiffusion-3, 102.4 for BindCraft, and 27.4 for BoltzDesign-1.

AuthorsOpenReview ↗

Reply to Reviewer pknh (Part II)

MethodBHRF1PDL1IRTrkAIL7RASC2RBDVEGFAH1IL17ATNFaAvg.
RFdiffusion37515425124071821319830370269
BoltzGen428549606804279342072481515318.5
Protpardelle-1c142403182489353110086.9
ODesign90137555948022412715211530293.6
PXDesign83712477567586952954641933157533.3
BoltzDesign-14254517317151345027.4
BindCraft12223122019489704128281102.4
ProteoCraft329412388410285220227711489248.1

Under the shared evaluation protocol and end-to-end computational accounting, ProteoCraft achieves both higher mean sequence-level pass rates and higher mean passing-backbone throughput than the hallucination-based baselines evaluated in this comparison.


We appreciate the specific suggestions that have helped improve the clarity and reproducibility of the manuscript. We hope the detailed responses resolve the ambiguities and welcome any further questions. If the clarifications and additional results above address your main concerns, we would be grateful if you would consider revising your score upward. Thank you again for your time and thoughtful feedback.

Reviewer_pXDhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ProteoCraft, a protein design framework that combines gradient-guided mutation proposals with tree search and full structure prediction. Approximate gradients from ESMFold2, together with an ESMC sequence prior, are used to propose mutations, while complete ESMFold2 inference evaluates the resulting sequences. The search proceeds in two stages: multi-site exploration followed by single-site refinement. The authors test the method on de novo binder design, TCR-facing binder design, and localized CDR redesign. They also evaluate the generated designs using AF2-IG and Rosetta, and report preliminary BLI binding experiments for IFNAR2 and PD-L1.

Critical strengths
  1. I like the separation between mutation proposals and structural verification. Rather than relying on approximate gradients to judge candidate quality, ProteoCraft uses them to suggest mutations and leaves the evaluation to complete structure prediction. This is a reasonable way to take advantage of gradient information without backpropagating through the full diffusion process. The coarse-to-fine search strategy also makes sense. Multi-site mutations allow broader exploration, while single-site refinement focuses on promising sequences. The mutable-mask formulation is useful for extending the approach to constrained design tasks such as CDR redesign.

  2. The results are fairly convincing in terms of predicted structural quality. On de novo binder design, MCTS improves mean Pair iPTM from 0.660 to 0.721 over Best-of-N, while the number of strict passes increases from 40 to 114 out of 210 designs (Table 1). The improvement is also seen on 13 of the 14 settings in Table 11. Importantly, the gains are not limited to ESMFold2. AF2-IG evaluation shows improved iPTM and iPAE, and several Rosetta interface metrics follow the same trend (Table 2). This makes the results more persuasive than improvements measured only by the model used during search. The preliminary BLI experiments are also a welcome addition, although the experimental evidence is still limited.

Critical weaknesses
  1. My main concern with the method is the incomplete specification of Stage B. Stage A has an explicit verification reward, but Stage B uses a different reward combining interface confidence, binder confidence, global confidence, compactness, and additional penalties. The exact formula, weights, and thresholds are not given in Section 2.4 or Appendix B.5. I also found the expansion procedure unclear. Section 2.4 describes logit optimization followed by verification of single-site candidates. Algorithm 1 appears to verify one mutation per Stage B expansion, whereas Table 7 lists up to eight candidates and two inserted children per expansion. Table 7 also specifies 20 soft optimization steps for Stage B, but this step is not explicit in the Stage B pseudocode. It is difficult to tell exactly which procedure was used. This matters because Stage B contributes a measurable improvement, increasing mean Pair iPTM from 0.691 to 0.721 over continued Stage A exploration. The authors should provide the complete Stage B reward and pseudocode consistent with the actual implementation. There are also some missing benchmark details. Table 8 does not include all the hotspot settings used in Table 11, including several DerF21 settings and DerF7-B178. The three binder lengths are not specified, and the AF2-IG Strict Pass criterion in Table 2 is not defined in Appendix C.2. These details should be straightforward to provide but are necessary for reproducing the results.

  2. I am less convinced by the efficiency comparison in Figure 1. First, BindCraft is missing from Figure 1, although it is the strongest external baseline by ESMFold2 Pair iPTM in both the de novo and TCR-facing benchmarks (Tables 1 and 3). The figure instead includes BindCraft v2, which performs considerably worse on de novo design. Table 12 also reports runtime for BindCraft v2 but not BindCraft. I would like to understand why the more competitive baseline was excluded. Second, I am not sure that the ProteoCraft and ProteoCraft-Fast points in Figure 1 are calculated using the same evaluation procedure. The ProteoCraft result appears to use the 210-candidate benchmark in Table 1, while Appendix C.3 evaluates accelerated configurations on a different set of 13 targets using best-of-five selection and multiple evaluation seeds. The authors should clarify how the quality values in Figure 1 were obtained. More importantly, the runtime comparison uses different definitions of a successful output. For ProteoCraft, the reported time is divided by the number of intermediate search candidates with overall iPTM above 0.70, rather than the number of final designs. Other methods are measured using generated outputs or their own filtering criteria (Appendix C.3). These numbers may be useful individually, but they do not provide a fair comparison of the time required to obtain a final design of comparable quality. Finally, Best-of-N and MCTS are described as having matched verification budgets, but the paper does not report the actual number of full verifier calls or the total number of soft optimization steps. Without these numbers, it is difficult to separate the benefit of tree search from differences in computational effort. I think the efficiency claims would be much stronger with a common output definition and explicit compute accounting.

Questions for authors
  1. Could the authors provide the exact Stage B reward function, including its coefficients and penalty thresholds? Please also clarify how many candidates are generated and verified per expansion, and whether the logits are reoptimized at every Stage B expansion. The pseudocode should agree with the configuration in Table 7.

  2. Why is BindCraft omitted from Figure 1 and the runtime tables? How are the ProteoCraft and ProteoCraft-Fast quality values calculated? Could the authors report the number of full verifier calls and soft optimization steps for Best-of-N and MCTS, and compare methods using a consistent definition of final output? This would have a substantial impact on my assessment.

  3. Could the authors provide the complete hotspot configurations, binder lengths, and AF2-IG Strict Pass criterion? I also noticed that BoltzGen has identical mean AF2-IG values in Tables 2 and 3 (iPAE = 26.342, iPTM = 0.200, pTM = 0.689), despite the different tasks. Could the authors check whether these values are correct?

  4. How many candidates were synthesized and tested for IFNAR2 and PD-L1, and how were the leads selected? The lead sequences, replicate information, and BLI fitting details would be helpful. In Figure 3, the PD-L1 trace at 1000 nM shows a lower response than several lower-concentration traces and a noticeably different dissociation profile. Could the authors explain this behavior and clarify how the reported K_D of 17.1 nM was obtained?

7 comments
AuthorsOpenReview ↗

Reply to Reviewer pXDh (Part I)

Thank you for your careful review of the method, evaluation, and experimental results. Your feedback helped us clarify the Stage B procedure, correct the BoltzGen statistics, and strengthen the quality–cost comparison. Below, we provide the complete Stage B specification, additional baseline measurements and computational accounting, corrected evaluation results, and expanded experimental details to address your concerns about reproducibility and experimental validation.

Q1. Complete Stage B Specification and Expansion Procedure

Thanks for your question regarding the soft optimization step. We take this opportunity to clearly distinguish the candidate-verification budget from the child-insertion budget.

At each Stage B expansion, we select a parent node and initialize soft optimization from its inherited sequence logits. We reoptimize these logits for 20 soft steps, construct a pool of up to 20 distinct single-residue substitutions, and rerank the candidates using a lightweight proposal-model forward pass. Up to eight candidates undergo complete ESMFold2 verification, with one exploration candidate included when sufficient alternatives are available. We then select up to two verified candidates for tree insertion, subject to the available expansion slots, and back up the rewards of the inserted actions. Thus, “single-site” describes the mutation in each candidate, while one expansion can evaluate several such candidates.

The Stage B reward is defined as

\frac{ \operatorname{clip}(1-I) + \operatorname{clip}(1-P) + \operatorname{clip}(1-T) + L_{R_g} }{4} + 0.25 L_{\mathrm{dist}} + 0.25 L_{\mathrm{cov}} + 2\,\mathbf{1}\{I < I_{\mathrm{floor}}\} \right]$$ where $\operatorname{clip}$ bounds a loss to $[0,1]$, $I$ is pair-chain iPTM, $P$ is binder pLDDT scaled to [0,1], and $T$ is global pTM. For binder length $L$, the compactness term is $$L_{R_g} = \operatorname{clip}_{[0,1]} \left( \frac{R_g}{2.38L^{0.365}} - 1 \right)$$ For a specified hotspot set $\mathcal H$, let $d_h$ denote the minimum heavy-atom distance from hotspot residue $h$ to the binder, in Å. We use $$L_{\mathrm{dist}} = \frac{1}{|\mathcal H|}\sum_{h\in\mathcal H}\min\left(1,\max\left(0,\frac{d_h-8}{12}\right)\right)$$ $$L_{\mathrm{cov}} = \frac{|\{h\in\mathcal H : d_h>8\}|}{|\mathcal H|}$$ Both terms are zero when no hotspots are specified. The additional fixed penalty for zero hotspot coverage has weight zero in the default configuration. Each branch inherits a quality floor from its reverified handoff root, $$I_{\mathrm{floor}} = \max(0, I_{\mathrm{handoff}} - 0.05)$$ A candidate below this floor receives a reward penalty of 2.0. For insertion, candidates meeting the floor are prioritized. Within this set, candidates with complete hotspot coverage are preferred, followed by greater coverage. For fully covered candidates, reward precedes hotspot distance in the ranking. For incompletely covered candidates, smaller mean hotspot distance precedes reward. If insertion slots remain, at most one below-floor candidate can be retained for exploration. We have revised the Stage B procedure in Alg 1. *** **Stage B — Single-site refinement** For $t=1,\ldots,B_B$: * Select an expansion parent $\mathbf{s}^i$ using PUCT and progressive widening. * Reoptimize its inherited logits $\boldsymbol{\Theta}^i$ for $K_B$ soft steps using the combined gradient. * Compute mutation scores $\Delta_{s_i,s_i'}$ and construct up to $K_{\mathrm{pool}}$ single-site candidates. * Rerank candidates and select up to $K_{\mathrm{verify}}$ for full verification. * Fully verify selected candidates with the common seed and archive all results. * Compute Stage B rewards and select up to $K_{\mathrm{child}}$ children using the quality-aware ranking. * Insert selected children with inherited, sequence-aligned logits. * Back up their state and action values. Return the best Stage B verified sequence and structure under the final quality-aware ranking. **Default settings** $K_A=30$, $K_B=20$, $K_{\mathrm{pool}}=20$, $K_{\mathrm{verify}}=8$, $K_{\mathrm{child}}=2$, and $N_{\mathrm{handoff}}=2$. The Stage B reward, quality floors, and ranking rules are specified in Appendix B.5.
AuthorsOpenReview ↗

Reply to Reviewer pXDh (Part II)

Q2. Baseline Coverage, Consistent Quality Evaluation, and Computational Accounting

(1) We agree that BindCraft v1 should be included because it is the strongest external baseline by ESMFold2 pair-chain iPTM in our original benchmarks. Including BindCraft v2 alone did not adequately represent this comparison. We have therefore measured BindCraft v1 runtime across the eight de novo targets, including its complete native pipeline of design, ProteinMPNN sequence design, and AF2/Rosetta filtering. At the current measurement cutoff, the runs have consumed 48.31 NVIDIA H200 GPU-hours and produced 74 designs passing the native filters, corresponding to a pooled cost of approximately 2,350 seconds per passing design. We have included BindCraft v1 in the revised Figure 1 and runtime table.

MethodDe novoTCR
PXDesign-d18.721.0
Proteina-Complexa14.634.6
RFdiffusion v356.644.2
BoltzDesign16932882
BindCraft2350—
BindCraft v22321—
ProteoCraft34.4174.0

(2) Thank you for identifying this ambiguity. The original presentation did not clearly distinguish unique protein targets from target–hotspot cases, which led to inconsistent aggregation across the figures and tables.

The de novo benchmark contains 14 target–hotspot cases across eight unique proteins, whereas the TCR-facing benchmark contains five cases across five unique targets. Figures 4 and 5 aggregate results over unique targets, giving 13 targets in total. Table 1 aggregates the de novo results over the fourteen target–hotspot cases. Consequently, the unique-target analysis gives a different weight to proteins represented by multiple hotspot settings.

We have rechecked the calculations and updated Figure 1 to use case-level aggregation consistently for ProteoCraft and ProteoCraft-Fast. Both sets of final candidates are evaluated using the same unaccelerated ESMFold2 20×20020\times200 configuration and the same candidate-selection and seed-aggregation procedure. The resulting mean pair-chain iPTM values are 0.721 and 0.702 for ProteoCraft and Fast on the 14 de novo cases, and 0.634 and 0.579 on the 5 TCR-facing cases.

Under this aggregation, the mean reported de novo runtime decreases from 34.4 s to 12.3 s per candidate, corresponding to 2.79× acceleration. The TCR-facing acceleration is 4.06×. When the de novo results are instead aggregated over the 8 unique proteins, the acceleration is 2.64×.

We will revise the captions and Appendix C.3 to state the evaluation cohort, aggregation unit, candidate-selection rule, and evaluation settings explicitly. Figures 4 and 5 will retain their unique-target analysis, with Figure 1 using case-level aggregation to align with the main benchmark.

(3) Thank you for requesting explicit computational accounting. Best-of-N and MCTS each receive 2,900 soft optimization steps and the same candidate-verification budget. MCTS additionally performs two handoff re-verifications, which we include in its total computational cost.

For Best-of-N, eight independent trajectories receive 6×370+2×340=2,9006\times370+2\times340=2{,}900 soft updates. Verification scores rank candidates but do not determine trajectory scheduling. For MCTS, Stage A receives 70×30=2,10070\times30=2{,}100 soft updates and Stage B receives 40×20=80040\times20=800, also totaling 2,900.

Both methods have a candidate-verification budget of 8+110×8=8888+110\times8=888 including eight initial-sequence evaluations. With its two additional handoff evaluations, MCTS has a full-verifier budget ceiling of 890 calls, compared with 888 for Best-of-N. These are budget ceilings; we will label them explicitly to distinguish them from realized execution counts.

We also use a consistent final-output definition. Both methods select final candidates using the same scoring rule and contribute 210 final designs from the same benchmark settings. These sequences undergo independent re-evaluation using unaccelerated ESMFold2 with the same 20×20020\times200 configuration. Intermediate search outputs are counted separately

Under this protocol, MCTS achieves mean pair-chain iPTM of 0.721 versus 0.660 and binder pLDDT of 0.917 versus 0.880. The number satisfying both pair-chain iPTM ≥0.75\geq0.75 and binder pLDDT ≥0.9\geq0.9 increases from 40/210 (19.0%) with Best-of-N to 114/210 (54.3%) with MCTS. These results demonstrate improved final-candidate quality under equal soft optimization budgets and closely matched verification budgets, with the two additional MCTS re-verifications fully accounted for.

AuthorsOpenReview ↗

Reply to Reviewer pXDh (Part III)

(4) Thanks for your question about the original runtime comparison. We would first clarify the meaning of an “intermediate search candidate” in ProteoCraft. Each archived candidate has a complete discrete sequence and has undergone full structure prediction. It can be independently evaluated, retained, and tested experimentally, regardless of whether the search subsequently expands or refines it. A proposal–verification run can therefore produce multiple usable designs. Counting these outputs captures the productivity of the search process, provided that they satisfy the same eligibility criterion applied to other methods.

To make output eligibility consistent across methods, we added a ProtDBench comparison under a measured end-to-end budget of 24 A100 GPU-hours, covering generation or search, ProteinMPNN sequence design, and independent AF2-IG evaluation.

All methods use the same output definition: a backbone qualifies when at least one of its eight ProteinMPNN sequences passes the shared independent AF2-IG filter. Each archived ProteoCraft candidate is a complete design that can be independently evaluated and retained, so such candidates count toward throughput only after passing this common evaluation, with their generation and evaluation costs included in the budget. Because your request was for GPU-hours per design passing the same independent threshold, we report yield under both the Easy and the Strict criterion rather than selecting one.

Under this protocol, the two criteria give different answers, and we think the difference is the informative part. Under Easy, ProteoCraft yields 247.9 passing backbones per 24 A100 GPU-hours, fifth of the eight methods. Under Strict, it yields 157.8, second of eight — behind PXDesign (299.8) and ahead of BoltzGen (148.1), RFdiffusion-3 (127.4), ODesign (73.9), BindCraft (53.8), Protpardelle-1c (11.8) and BoltzDesign-1 (5.6). We achieve the best performance among hallucination-based approaches.

MethodStrict CriteriaRankEasy CriteriaRankRetention
PXDesign299.81533.3156.2%
ProteoCraft157.82248.1563.7%
BoltzGen148.13318.5246.5%
RFdiffusion-3127.44269.0447.4%
ODesign73.95293.6325.2%
BindCraft53.86102.4652.5%
Protpardelle-1c11.8786.9713.6%
BoltzDesign-15.6827.4820.4%
MethodBHRF1PDL1IRTrkAIL7RASC2RBDVEGFAH1IL17ATNFaAvg.
RFdiffusion37515425124071821319830370269
BoltzGen428549606804279342072481515318.5
Protpardelle-1c142403182489353110086.9
ODesign90137555948022412715211530293.6
PXDesign83712477567586952954641933157533.3
BoltzDesign-14254517317151345027.4
BindCraft12223122019489704128281102.4
ProteoCraft329412388410285220227711489248.1
MethodBHRF1PDL1IRTrkAIL7RASC2RBDVEGFAH1IL17ATNFαAvg.
RFdiffusion-346222528521607214000127.4
BoltzGen3652523494560057002148.1
Protpardelle-1c59530600000011.8
ODesign36298192630131200073.9
PXDesign685100142350021167193208299.8
BoltzDesign-12168270110005.6
BindCraft70168135126128722053.8
ProteoCraft27335534232344147780106157.8

The reordering is explained by how much of each method’s Easy-criterion yield survives the Strict filter. ProteoCraft retains 63.7%, the highest in the panel, against 56.2% for PXDesign, 52.5% for BindCraft, 47.4% for RFdiffusion-3, 46.5% for BoltzGen, 25.2% for ODesign, 20.4% for BoltzDesign-1 and 13.6% for Protpardelle-1c. This is the behavior we would expect if verification-guided search spends its budget raising individual candidates past a higher bar rather than enlarging the candidate pool: the advantage is modest when the bar is low and grows as it rises. Against BindCraft, the closest hallucination-based comparator, the ratio widens from 2.42× under Easy to 2.93× under Strict.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Revisions and Evidence for the Area Chair and Reviewers

Dear Area Chair and Reviewers,

Thank you for the detailed and constructive feedback. We have expanded prospective experimental validation, added a standardized ProtDBench comparison with measured A100 computational costs, directly evaluated mutation proposals, and added a beam-search baseline. Below, we summarize the new evidence and the clarifications that will accompany the revision.

1. Wet-lab experiments directly establish functional binding among tested designs.

For PD-L1, we evaluated 35 designed miniproteins, all of which passed expression and loading quality control. Twenty-nine exhibited measurable binding, yielding an observed hit rate of 82.9%. Six candidates had fitted KDK_D values below 10 nM, including a best-fitted affinity of 0.879 nM. This value derives from a single SPR measurement, which we explicitly disclose.

For IFNAR2, four of nine successfully expressed VHH candidates and five of 27 successfully expressed miniproteins showed binding, corresponding to expression-conditioned hit rates of 44.4% and 18.5%, respectively. The best-fitted affinity is 26.29 nM. These results provide prospective evidence across two targets and two binder formats. We will include complete candidate accounting, selection procedures, sequences, and measurement details in the revised presentation.

We respectfully ask that this experimental evidence receive substantial weight in assessing practical design performance. Structure-prediction confidence, independent predictors, and Rosetta provide valuable computational evidence, while binding assays directly test the biological objective. Our retrospective analysis of 1,230 experimentally tested Anthropic designs illustrates the remaining uncertainty in computational screening. Across individual predictors, top-10% ipSAE selection yields reported hit rates of 48.8%–61.9% on the 12 targets with at least five positives. Confidence ranking enriches experimental hits, but substantial uncertainty remains even among highly ranked candidates.

2. The additional ProtDBench evaluation broadens target coverage and quantifies computational trade-offs.

We evaluated ProteoCraft on all ten ProtDBench targets at five matched binder lengths of 80, 90, 100, 110, and 120 residues, using a shared ProteinMPNN and independent AF2-IG evaluation procedure.

ProteoCraft achieves mean Easy and Strict sequence pass rates of 54.50% and 27.06%, exceeding the strongest corresponding baselines by 20.62% and 15.13%. It achieves the highest Easy pass rate on nine of ten targets and the highest Strict pass rate on nine targets. All methods have zero Strict passes on H1.

We also report passing-backbone yield per 24 A100 GPU-hours, accounting for generation or search, sequence design, and independent evaluation, under a shared acceptance rule applied identically to every method. Under a shared acceptance rule and measured A100 budgets, ProteoCraft yields 247.9 passing backbones per 24 GPU-hours at the Easy threshold and 157.8 at the Strict threshold. It is the strongest hallucination-based method under both criteria, by 2.4x over BindCraft under Easy and 2.9x under Strict. Against the diffusion baselines the gap also closes as the bar rises: four of them exceed ProteoCraft under Easy, but only PXDesign (299.8) does under Strict. ProteoCraft moves from fifth to second of eight methods as the criterion tightens and has the highest Easy-to-Strict retention in the panel (63.7%). PXDesign leads on mean yield under both criteria; the trade-off between ProteoCraft's higher sequence pass rates and its search cost is explicit.

Structural clustering using ProtDBench’s TM-align procedure at a TM-score threshold of 0.6 gives ProteoCraft a mean cluster pass rate of 17.5%. This exceeds the listed diffusion baselines’ 4.0%–14.7%, is close to BoltzDesign-1’s 18.4%.

3. New controls directly assess gradient proposals and alternative search strategies.

The existing No One-step Diffusion ablation retains distogram and ESMC guidance. Before tree search, adding one-step diffusion increases mean pair-chain iPTM from 0.637 to 0.660 and the number of candidates above 0.7 from 85/210 to 118/210.

We additionally evaluated 128 single-residue substitutions per strategy, comparing random, ESMC-only, distogram-only, distogram+ESMC, and full proposals through complete structure prediction. At 20 soft updates, the full method achieves the largest observed mean pair-chain iPTM improvement, +0.004560, compared with +0.002096 for random and +0.003458 for distogram+ESMC. Improvements exceeding 0.01 occur for 23/128 full proposals and 13/128 random proposals, a difference of 7.8 percentage points with a paired 95% confidence interval of +0.8 to +16.4. The incremental difference from distogram+ESMC remains uncertain.

AuthorsOpenReview ↗

Summary of Revisions and Evidence for the Area Chair and Reviewers

To test an alternative search strategy, we added ProteoCraft-BeamSearch on the original 210-design benchmark. MCTS achieves a mean ESMFold2 pair-chain iPTM of 0.721 versus 0.638 for beam search. Under independent AF2-IG evaluation, Strict Pass increases from 55/210 (26.2%) with beam search to 181/210 (86.2%) with MCTS. Rosetta interface evaluation also favors MCTS.

For Best-of-N and MCTS, we now specify 2,900 soft optimization steps per method and full-verifier budget ceilings of 888 and 890 calls, respectively. The two additional MCTS calls are handoff re-verifications and are included in its cost. Both methods contribute 210 final candidates selected under the same scoring rule and independently reevaluated using the same unaccelerated ESMFold2 configuration.

4. Additional predictors and clearer reporting improve reproducibility.

We evaluated the original 210 designs per method using OpenDDE and Protenix-v2 under shared sequence-only prediction settings. ProteoCraft-MCTS achieves mean ipTM/pTM values of 0.6040/0.7092 with OpenDDE and 0.5703/0.6887 with Protenix-v2, exceeding the external baselines and the No One-step Diffusion control included in these comparisons.

We will revise the Stage B description to provide its complete reward, coefficients, penalty thresholds, and pseudocode, explicitly distinguishing 20 soft updates, up to 20 proposals, up to eight full-verifier evaluations, and up to two inserted children. We will also complete the hotspot configurations, binder lengths, and AF2-IG acceptance criteria.

For Figure 1, we have clarified the distinction between unique-target and target–hotspot-case aggregation, updated the Fast calculations, and added BindCraft v1 runtime measurements. We will explicitly identify method-specific output criteria and distinguish these timings from the shared-criterion ProtDBench comparison.

Finally, we acknowledge the duplicated BoltzGen statistics identified by a reviewer. We have reevaluated all 210 original de novo BoltzGen sequences, preserving the candidate set, and will correct the affected table and provide the evaluation records.

We have re-verified every table end-to-end against the raw evaluation records and will release the per-design evaluation records with the code.


We appreciate the reviewers’ suggestions, which have helped strengthen the experiments and improve the manuscript’s clarity. We respectfully ask the Area Chair and reviewers to consider the expanded prospective binding evidence, standardized computational comparisons, and new methodological controls in their final assessment.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.