acceptodds
Under review as a conference paper at ICLR 2027

Understanding and Improving Unique Hit Scaling in Protein Binder Design

Abstract

Generative models are accelerating the discovery of proteins that bind specific molecular targets, creating new opportunities for therapeutic development. The goal of an in silico campaign is to propose a diverse library of promising candidates for experimental testing, motivating the objective of maximizing unique hits under a fixed compute budget. However, how unique hit scales with compute remains poorly characterized, and methods focused on improving hit rate have produced mixed results. We propose a cluster-level view of binder design that decomposes unique hit scaling into two processes: discovering structural clusters through generation and discovering hits within those clusters through evaluation. This decomposition provides a predictive model of campaign yield and exposes a new axis for scaling: allocation of compute between generation and evaluation. We introduce a cluster-guided allocation (CGA) policy that estimates two processes online and allocates compute according to the expected unique hit yield per unit cost. Across 19 targets and three generators, the two-process decomposition characterizes the campaign well, and CGA outperforms naive sampling, showing consistent and predictable gains not seen in previous reward-tilting methods. Beyond these gains, the framework provides a mechanistic understanding of binder design scaling and reveals complementary opportunities.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
protein binder design, protein generative models

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_aqWkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates protein binder design as a compute-allocation problem, with the objective of finding structurally distinct in silico hits under a limited budget. The authors separate the discovery of structural clusters from the discovery of hits within those clusters. CGA estimates both processes online and uses them to decide when to generate more candidates, when to evaluate existing candidates, and which clusters to prioritize. This framework incorporates information accumulated throughout the campaign into subsequent decisions.

Critical strengths

What I find most valuable is the paper’s reconsideration of the optimization objective. A candidate’s value depends on what the campaign has already discovered. Once a cluster has produced a hit, another hit from the same cluster adds nothing to the objective. Maximizing sample-level hit probability alone does not fully capture this diminishing marginal value. This explains why a higher hit rate need not produce more unique hits. Incorporating this distinction directly into the decision objective identifies a meaningful way to improve a campaign: keep the generator unchanged and improve how compute is allocated.

The analysis of failure information is also insightful. Equation 5 relates the reduction in next-hit probability to the remaining uncertainty about cluster quality. If all clusters have the same hit probability, previous failures provide no reason to lower the priority of a particular cluster. When cluster qualities differ, repeated failures gradually change our assessment of a structural region. This analysis clarifies when moving away from an unsuccessful region is justified. It also connects the allocation policy to the statistical properties of a particular generator–target combination.

Critical weaknesses

My main reservation concerns the assumption that a cluster’s accessibility to the generator is independent of its quality. Under Appendix B.2, how frequently the generator visits a cluster provides no information about that cluster’s probability of producing a hit. Consequently, all unevaluated clusters receive the same initial quality prediction, regardless of how many candidates they contain.

This assumption can materially affect the decisions. If the generator’s preference for a structural region carries information about quality, CGA discards information that should influence evaluation priorities. For example, frequently generated and rarely generated clusters receive the same predicted hit prospects before any evaluation. Two aspects need to be distinguished here. Separating structural discovery from hit discovery is useful; fitting them as independent processes further restricts the phenomena the model can explain. I would like the authors to clarify the scope of this approximation.

The algorithmic contribution could also be stated more clearly. With a shared quality prior and equal evaluation costs, the evaluation order reduces to least-failures-first regardless of the particular prior. The fitted probability distributions mainly affect when to generate more candidates and the prediction of final yield. I would therefore like a fuller explanation of the generation–evaluation switching rule: how do the probability of discovering a new cluster, the hit prospects of existing candidates, and the costs of both operations jointly determine this choice? This deserves more attention than the connection to classical multi-armed bandit indices and would better bring out the method’s distinctive contribution.

Questions for authors

Why is cluster generation frequency treated as uninformative about cluster quality? Is this a simplifying approximation, or do the authors consider it a property of these generators? If it is a modeling approximation, please clarify which explanatory conclusions depend on it. A relationship between accessibility and quality would affect both evaluation priorities and the interpretation of the fitted scaling model.

6 comments

Thank you for recognizing the value of our objective and perspective. We also appreciate your questions about the assumption and the generation-evaluation rule, which motivated additional analyses and helped us clarify our algorithmic contribution.

Why is cluster generation frequency treated as uninformative about cluster quality?

We treat independence as a simplifying approximation motivated by our observations from naive campaigns. Larger clusters show little enrichment overall, and the relationship between cluster size and quality varies across campaigns, as shown in this plot*. Within each campaign, we grouped clusters by size quartile and measured hit enrichment relative to the campaign’s overall hit rate. Thin gray lines show individual campaigns, while the thick colored line shows the mean enrichment with equal weight assigned to each campaign.

The largest size quartile has mean hit enrichment of 0.94×, 1.07×, and 1.03× for Complexa, Genie 3, and RFDiffusion3, respectively. But because the direction and strength of enrichment vary across generator-target pairs, the relationship has to be learned rather than using one fixed rule. We therefore chose not to model this dependence in the current formulation to avoid additional complexity. Studying this relationship across a larger target set, and understanding why it differs between generators and campaigns, is an interesting future direction that could inform the development of both better allocation policies and generators.

*As in Fig. 3, the comparison is restricted to generator-target pairs with at least 20 observed hits.

Fuller explanation of the G-E rule: how do the probability of discovering a new cluster, the hit prospects of existing candidates, and the costs of both operations jointly determine this choice?

Thank you for this constructive feedback. Generation and evaluation indices calculate expected additional unique hits per unit compute. Evaluating uses a candidate already available, whereas generating incurs an additional cost to obtain a potentially more promising candidate. This tradeoff is ultimately a function of the property of generator-target pair characterizing the campaign (e.g. Is this generator diverse? overall high hit rate?) and generation-to-evaluation cost ratio, cG/cEc_G/c_E.

For each campaign state, there will be a break-even cost ratio at which the generation index equals the best available evaluation index. Generation is preferred below this ratio. A larger break-even ratio therefore means that generation remains worthwhile even at a higher cost.

We created an interactive visualization (Please download here) covering combinations characterizing campaigns: high or low diversity, heterogeneous or homogeneous cluster quality, and high or low overall hit rates. A slider varies cG/cEc_G/c_E while holding the selected campaign state fixed, showing how the indices and preferred action change. The table below summarizes these examples, ordered by their break-even ratios. The thresholds and ranking are specific to the illustrated states in the visualization. We will include the visualization in the project page when we release the paper.

(continued in next response)

DiversityCluster qualityOverall hit rateWhy the policy favors G or EcG/cE break-even ratio
HighHeterogeneousLowG has the strongest advantage. Successful clusters hit early and retire, while repeated failures lower the prospects of the clusters still available. These uneven outcomes suggest that fresh clusters can be substantially better than the remaining pool. Higher discovery probability makes those fresh opportunities easier to reach, allowing generation to justify a larger cost.0.3602
LowHeterogeneousLowSame as 1, fresh clusters can offer substantially better hit prospects. However, most generations revisit existing clusters, so generation must be cheaper to justify the search.0.2269
HighHomogeneousLowE is preferred unless generation is inexpensive. Repeated failures across every cluster lower expectations for fresh clusters too. A higher discovery probability makes fresh opportunities easier to reach, but their quality advantage must still justify the extra generation cost.0.0983
HighHeterogeneousHighG offers better prospects, but E still has useful candidates. High diversity makes fresh clusters more accessible, while the high overall hit rate supports promising fresh clusters. However, available clusters with low failure counts still offer useful candidates. Generation must be sufficiently cheap to justify searching for better prospects instead of evaluating those candidates directly.0.0672
LowHomogeneousLowSame as 3, but discovery is even less likely.0.0547
LowHeterogeneousHighSame as 4, but discovery is even less likely.0.0334
Low/HighHomogeneousHighE wins at any positive generation cost. Observed successes support high hit prospects, and an available candidate already matches the fitted prospects of a fresh draw. Generation adds acquisition cost without improving hit prospects, regardless of diversity. Free generation only ties direct evaluation.0

Our algorithmic contribution is an adaptive allocation rule that combines estimated cluster discovery and quality with generation and evaluation costs to decide which operation to perform next. The rule responds to changing opportunities throughout the campaign and empirically improves unique hit yield under a fixed compute budget across a range of generator–target pairs.

I thank the authors for the detailed response and the additional analyses.

On the independence assumption. The quartile-level enrichment ratios are helpful and I appreciate the honesty in characterizing this as a simplifying approximation. However, my concern is less about whether the marginal correlation is strong and more about whether CGA's decisions are sensitive to even mild violations. The enrichment ratios show that the relationship varies in both direction and strength across campaigns, which means the approximation is better for some campaigns than others. A brief experiment showing how CGA's yield changes when a small positive or negative correlation between accessibility and quality is introduced would go a long way toward bounding the practical impact of this assumption. Without such a check, I cannot fully assess whether the method's improvements come partly from regimes where the assumption holds well and are offset in regimes where it does not.

On the G/E switching rule. The break-even cost ratio table and interactive visualization are genuinely clarifying. This analysis makes the algorithmic contribution much more concrete and I encourage the authors to incorporate it into the camera-ready version. I note that the table also reveals that generation is never preferred over evaluation when cluster quality is homogeneous and the hit rate is high (break-even ratio of 0). This is a useful boundary condition that deserves explicit discussion, as it helps practitioners understand when CGA offers the most value over simpler strategies.

Overall, this rebuttal strengthens my understanding of the method and partially addresses my concerns. The G/E analysis is a meaningful addition. The independence assumption remains an open limitation whose practical impact is not yet bounded. I maintain my score of weak accept.

Reviewer_H5RXOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The author proposed a method to improve unique hits during binder-generation campaigns by focusing on the cluster center.

Critical strengths
  1. Evaluation of the sequences of the cluster center accelerates in silico binder screening for a massive binder design campaign.
Critical weaknesses
  1. The benefit of the full statistical framework over simpler screening policies needs stronger evidence. The retire-on-hit comparison covers only four Complexa targets, so its additional value across generators is not well established.
Questions for authors
  1. CGA leaves the base generator unchanged, so finding new clusters still depends on that model. What happens when it keeps generating the same folds, or when most new clusters fail the filters?

  2. You use “one viable binding mode” to explain cluster quality, but binder folds and binding modes are not the same. Are you clustering binder structures, full complexes, or interfaces? Why should these clusters represent different binding solutions?

  3. You acknowledge that the predictor scores are uncalibrated, but binarizing them does not fix this. Do the gains hold under different hit cutoffs, and how often do those changes alter which clusters get retired?

  4. The extra clusters are supposed to provide more useful candidates. How do you know they are meaningful binding alternatives rather than outliers that pass the predictor? Is there any independent validation of the additional hit clusters?

  5. Beam/FK favor predictor scores rather than structural diversity, and FK uses a temperature of 0.1 on standardized rewards. Do they still underperform with weaker steering or diversity-aware rewards, or is the result specific to these settings?

  6. The TM-score cutoff analysis is helpful, but did you inspect the actual structures? Can you show whether the additional clusters have different binding poses or interface contacts, beyond differences in the overall binder fold?

  7. How are cluster representatives chosen, and which member actually gets evaluated? Since memberships can change, how do you update previous failures and retirement decisions when clusters are reassigned?

6 comments

Thank you for your constructive feedback and questions, which led to valuable additional analyses.

Clarification on the stated summary and strengths of our work

We first clarify that we do not restrict evaluation to cluster centers: any candidate in an unhit cluster may be evaluated. Our contribution is to use clustering throughout the campaign (as opposed to current practice that defers clustering to the end of the campaign, as a post-filtering step) to guide compute allocation. We propose models of how generation discovers structural clusters and how evaluation finds hits within them, and these models enable forecasts of campaign yield. We hope this clarification addresses the concern and better conveys our framework’s contribution to understanding and improving binder design campaigns.

Q1. CGA leaves the base generator unchanged, so finding new clusters still depends on that model. What happens when it keeps generating the same folds, or when most new clusters fail the filters?

We distinguish the policy’s behavior in these cases from its empirical performance.

  • Policy behavior. CGA cannot discover folds that the generator cannot produce. In the extreme case where all generated candidates belong to a single cluster, there is no advantage from choosing among clusters. More generally, when generation repeatedly revisits several folds, CGA can save evaluation compute by skipping candidates from clusters that have already produced a hit. When most clusters fail, the relevant distinction is whether their hit probabilities differ. If repeated failures indicate that an existing cluster is unlikely to succeed, generation becomes more attractive when it can discover more promising clusters. If clusters instead have similarly low hit probabilities, seeking a better cluster offers less benefit, and evaluating available candidates becomes comparatively more attractive. Low hit rate alone does not determine the allocation. The decision depends jointly on the fitted discovery process, cluster quality distribution, and generation and evaluation costs.
  • Empirical performance and interpretation. Low yield (i.e. most clusters fail) does not necessarily eliminate the benefit of allocation. On the low-yield target group, CGA produces 2.59× and 1.25× as many unique hits as naive sampling for Complexa and RFDiffusion3, respectively, although gains are not universal: Genie 3 yields 0.94× (Sec. 4.3).

Our framework also provides a common basis for comparing generators by predicting campaign outcomes. The symptoms highlighted in the question map directly onto our two models: “keeps generating the same folds” is captured by the cluster discovery process, and “most new clusters fail the filters” is captured by the cluster quality distribution. Once fitted, these models allow us to forecast unique hit yield under a given compute budget, without further calls to the generator or structure predictor. These forecasts could inform the choice of generator expected to yield the most unique hits for a particular target. Extending the framework from allocating compute within a fixed generator to choosing among generators is an exciting direction for future work.

Q2. You use “one viable binding mode” to explain cluster quality, but binder folds and binding modes are not the same. Are you clustering binder structures, full complexes, or interfaces? Why should these clusters represent different binding solutions?

We agree that binder folds and binding modes are not equivalent. We cluster binder structures, using Foldseek with a TM-score cutoff of 0.6. The example was intended to illustrate the cluster quality distribution GG. We have replaced "binding mode" with "binder fold" in the revised manuscript.

Q3. You acknowledge that the predictor scores are uncalibrated, but binarizing them does not fix this. Do the gains hold under different hit cutoffs, and how often do those changes alter which clusters get retired?

We agree that binarizing predictor scores does not calibrate them against experimental outcomes. Binarization defines the campaign’s success criterion. We use a conventional hit definition that prior studies have shown to enrich for experimentally successful binders; relaxed or tightened cutoffs cannot be assumed to provide the same enrichment.

Q4. The extra clusters are supposed to provide more useful candidates. How do you know they are meaningful binding alternatives rather than outliers that pass the predictor? Is there any independent validation of the additional hit clusters?

CGA and naive sampling use the same unsteered generator; CGA changes compute allocation and candidate evaluation decisions. We compared the distributions of predictor scores and independent interface quality metrics between hits obtained by CGA and naive sampling. These interface quality metrics, also used in the Complexa paper, are not used to define hits or guide CGA’s allocation decisions. These additional metrics therefore provide independent in silico validation beyond the scores used to select hits.

The distributions overlapped closely, as shown in this figure. Across 33 matched generator-target pairs with at least 30 hits under each method, the median absolute differences between the methods’ per-pair median scores were only 0.012 Å for scRMSD, 0.003 for binder pTM, and 0.015 Å for minimum interface PAE. The corresponding differences were 0 for interface hydrogen-bond count, 0.004 for interface shape complementarity, 23.2 Ų for total buried interface SASA, and 0 for the number of binder interface residues.

These results suggest that CGA increases unique hit yield while preserving hit quality, as assessed by both predictor scores and independent interface metrics.

Q5. Beam/FK favor predictor scores rather than structural diversity, and FK uses a temperature of 0.1 on standardized rewards. Do they still underperform with weaker steering or diversity-aware rewards, or is the result specific to these settings?

Our baseline hyperparameters follow the Complexa paper. We additionally tested a diversity-aware beam-search reward that uses cluster counts within the current beam run. The table below shows unique hit count at 22 hour, using Complexa as generator. The diversity reward produces only small and inconsistent changes in hit counts across the four tested targets.

TargetStandard beamDiversity 0.03Diversity 0.1
TrkA858590
PD-1828485
IL7RA474644
CrSAS6121212

Implementation detail.

The diversity-aware reward is Ri=−interface PAEi31+λ1+Nc(i)+Sc(i)⏟diversity bonusR_i = -\frac{\text{interface PAE}_i}{31} + \underbrace{\frac{\lambda}{\sqrt{1 + N_{c(i)} + S_{c(i)}}}}_{\text{diversity bonus}}

  • λ\lambda: 0.03 or 0.1 controls the diversity reward weight
  • c(i)c(i): candidate ii’s Foldseek structural cluster at TM 0.6.
  • NcN_c: diversity across checkpoints. If many candidates from cluster cc were evaluated at earlier checkpoints, its candidates receive a smaller bonus now. This favors exploring less-visited clusters within the current beam run.
  • ScS_c: diversity among the current checkpoint. Each time a candidate from cluster cc is selected, ScS_c increases by one, reducing the bonus for remaining candidates from that cluster. This encourages the four survivors to come from different clusters.

We believe that the value of our work extends beyond the specific reward-tilting settings. Its gains come from using campaign history to avoid redundant evaluations, prioritize promising unhit clusters, and balance generation against evaluation. Even with a diversity-aware reward, steering alone does not explicitly model the campaign at the cluster level or determine when compute is better spent generating new candidates versus evaluating available ones.

Q6. The TM-score cutoff analysis is helpful, but did you inspect the actual structures? Can you show whether the additional clusters have different binding poses or interface contacts, beyond differences in the overall binder fold?

We first clarify that our diversity objective intentionally includes differences in the overall binder fold, preserving designs that may satisfy downstream requirements not known in advance. These requirements may concern the interface or properties influenced by the binder scaffold, such as stability, solubility, and compatibility with interacting partners.

To assess interface diversity more directly, we additionally incorporated interface contact similarity into clustering. For each complex, we defined the contact set as target residues with at least one heavy atom within 4.5 Å of a binder heavy atom, and compared these sets using Jaccard similarity. The gains persist when interface-contact similarity is incorporated: CGA yields 1.51×, 1.08×, and 1.06× as many hit clusters as naive sampling for Complexa, Genie 3, and RFDiffusion3, respectively. The results grouped by low, medium, high-yield target groups are shown in the radar chart.

We also visualize structures here, showing which binder folds get clustered together, and when does interface contacts cluster them apart in this new analysis.

Q7. How are cluster representatives chosen, and which member actually gets evaluated? Since memberships can change, how do you update previous failures and retirement decisions when clusters are reassigned

Cluster representatives are selected by Foldseek’s greedy set-cover clustering procedure and are used for efficient structural assignment. Evaluation is not restricted to representatives: any unevaluated member of a currently unhit cluster is eligible, and CGA evaluates one available candidate from each selected cluster. Once a cluster contains a hit, its remaining candidates are excluded from evaluation while that cluster remains retired.

Evaluation outcomes are stored on individual designs, rather than permanently attached to clusters. When cluster assignments change, we reconstruct each cluster’s statistics from its current membership:

  • Failure count: the number of current members that were evaluated and failed the hit criteria.
  • Retirement status: whether any current member is an evaluated hit.

W1. CGA vs retire-on-hit comparison across generators

Please check our response to Reviewer PVyt. The expanded ablation in all 3 generators and 12 targets confirms that retire-on-hit component alone does not account for CGA’s gains. Relative to retire-on-hit only, CGA increases aggregate unique hit yield by 35.6%, 20.8%, and 3.8% in the low-, medium-, and high-yield groups, respectively.

Reviewer_PVytOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Protein binder generation campaigns aim to maximize the number of unique hits given a certain compute budget. This budget is split between generating candidates (either naive sampling, or some search algorithm) and evaluating candidates with a folding model to determine which ones are "hits", that is, which ones pass the success criteria. In general, a high raw hit rate is not the end goal: The performance of different methods is typically measured as the number of hits that are different from each other; formally, the unique pass rate. This involves taking all hits, clustering them, and reporting the number of different clusters.

Despite this, most work in this domain uses different methods and simply evaluates every generated candidate. This work identifies this as suboptimal: given a successful sample, all other generated samples that are similar to it (ie. fall in the same cluster) do not need to be evaluated, saving some of the computational budget typically spent doing so. This work addresses this by developing a policy that alternates between generation and evaluation with the goal of maximizing unique successes. They show that, under a fixed computational budget, they can improve Proteina-Complexa unique pass rate by ~60%, and RFDiffusion3 and Genie3 by ~15%. This is achieved by simply changing the selection mechanism used to determine what samples to evaluate.

Critical strengths

This paper tackles the problem of increasing the efficiency of in-silico protein design campaigns. This is a relevant problem. As far as I know, most work in this space attempts this by improving the generative model to have a higher hit rate, or by improving the sampling / evaluation efficiency. This work takes an alternative approach by developing a method that's applicable to any generator, and finds a policy that attempts to maximize the number of unique hits in a given computational budget. This is generic, and the formulation in terms of probabilistic modeling is elegant. For some models the approach yields ~60% improvements over the naive approach at a negligible computational cost.

Critical weaknesses

Gains, while very good for Complexa, are somewhat limited for the other two models tested. In current formulation, if the base model is very diverse, this approach is less helpful. In fact, if each sample falls in its own cluster (very diverse design model), then the resulting policy is simply to evaluate all samples, which coincides with the naive approach. The method is therefore most efficient when the generator used tends to generate "uneven clusters", where some clusters are overly populated and several others receive just a few samples. In this case (like Complexa) a lot of compute can be saved simply by not evaluating samples in a given cluster which already has a hit. For other models which appear to spread out design across clusters more evenly (like RFDiffusion3 and Genie3), the method appears to be less helpful.

The quality model uses only binary evaluation outcomes. Consequently, unhit clusters with the same number of failures receive the same predicted hit probability, even when their continuous evaluation metrics differ substantially. This discards potentially useful information about how close candidates are to satisfying the success criteria. A model incorporating these metrics could distinguish promising near-miss clusters from consistently poor ones. The paper acknowledges this extension, but an analysis of whether continuous score histories provide predictive information beyond failure counts would help assess the consequences of the binary-only assumption.

Questions for authors

Does the retire-on-hit baseline in Appendix D.3 use least-failures-first to select evaluations? Please clarify its selection rule. Also, can you report results for RFD3 and Genie3 for this baseline? It would be interesting to see how much of the performance improvement for them can be achieved with the simple baseline. Additionally, would it be possible to report results across all targets, for the three methods, using the following policy parameterized by K:

1- Generate K samples

2- Cluster existing samples

3- Evaluate one sample in the unhit clusters with less failures so far

4- If no eligible clusters/candidates remain, generate more

5- Go to step 1

As far as I understand, this is somewhat similar to your retire-on-hit baseline, but parameterized by K, and for all methods. I think seeing these results for K in {1, 2, 4, 8} would be informative regarding the method's benefits. The ablation for Complexa on 4 targets is encouraging, but this question seems unresolved across the full target set, even for Complexa. Evidence that CGA provides meaningful additional benefits under these comparisons would lead me to reconsider my score.

5 comments

We thank the reviewer for recognizing the general applicability and elegant probabilistic formulation of our approach, and for the constructive feedback. In response, we clarify the retire-on-hit baseline and expand our ablations to all three generators and 12 targets, separating the contributions of retirement, least-failures-first evaluation, and adaptive generation-evaluation allocation. We also clarify how structural diversity and cluster quality affect CGA’s gains, and add a preliminary ablation examining the value of continuous evaluation scores.

CGA vs retire-on-hit plus least-failures-first with a simple generation–evaluation schedule (Q1)

The retire-on-hit baseline in Appendix D.3 selects randomly among clusters that have not yet produced a hit, without using least-failures-first to prioritize those with fewer failed evaluations. We have expanded this ablation to all three generators, including RFD3 and Genie3, and 12 of the 19 targets, with four targets from each of the low-, medium-, and high-yield groups. We also evaluated your proposed fixed generation-evaluation policy incorporating both retire-on-hit and least-failures-first for $K \in { 1,2,4 } $. The expanded experiments required 2,592 H200 GPU-hours; Extending the evaluation to all 19 targets and including K=8K=8 exceeded the compute available during the rebuttal period. However, the 12-target evaluation spans a broad range of campaign yields across all three generators. We now compare the following policies:

NameRetire-on-hitLeast-failures-firstGenerate vs Evaluate
NaiveNoNo1:1
Retire-on-hit onlyYesNo1:1
GE (new)YesYes1:1
GGE (new)YesYes2:1
GGGGE (new)YesYes4:1
CGA (ours)YesYesAdaptive decision, via proposed index

We uploaded the full per-target unique hit yield CSV and radar chart visualization to this anonymous link: https://anonymous.4open.science/r/rebuttal-scaling-0666/ablation_full_results.csv, https://anonymous.4open.science/r/rebuttal-scaling-0666/ablation_radar.pdf. Below, we summarize the results.

First, the expanded ablation in all 3 generators and 12 targets confirms that retire-on-hit component alone does not account for CGA’s gains. Relative to retire-on-hit only, CGA increases aggregate unique hit yield by 35.6%, 20.8%, and 3.8% in the low-, medium-, and high-yield groups, respectively.

Second, comparing CGA with GE, GGE, and GGGGE assesses the added value of CGA’s third component (using fitted probability models to adaptively choose between generation and evaluation) as requested by several reviewers. All three schedules are a strong baseline because it already incorporates our central idea of cluster-aware allocation and two of CGA’s three components: retire-on-hit and least-failures-first. The table below reports fold improvements over naive sampling, also visualized in the radar chart. The best policy for each yield group is bolded, and the second-best is underlined.

GroupGeneratorGEGGEGGGGECGA
LowComplexa1.33×1.71×2.54×2.25×
LowGenie 30.67×0.78×0.78×0.78×
LowRFdiffusion30.87×1.11×0.83×1.27×
LowAll three0.97×1.23×1.25‾\underline{1.25}x1.47×
MediumComplexa1.17×1.53×1.78×1.70×
MediumGenie 31.21×1.17×0.92×1.21×
MediumRFdiffusion31.04×1.15×1.15×1.31×
MediumAll three1.16×1.31‾\underline{1.31}×1.30×1.42×
HighComplexa1.42×1.61×1.82×1.57×
HighGenie 31.13×1.12×0.87×1.12×
HighRFdiffusion31.13×1.23×1.13×1.18×
HighAll three1.18×1.24×1.11×1.22‾\underline{1.22}x

The variation in fixed-schedule performance across groups justifies adaptive allocation: increasing the generation-evaluation ratio from 1:1 (GE) to 4:1 (GGGGE) improves performance for high-yield Complexa from 1.42× to 1.82× over naive, but reduces it for high-yield Genie 3 from 1.13× to 0.87×. Thus, a schedule that benefits one campaign can hurt another, highlighting the value of adapting this ratio to each campaign.

CGA delivers consistently strong performance across groups, achieving the highest aggregate improvement in the low- and medium-yield groups and remaining close to the best fixed schedule in the high-yield group.

Thanks again for the valuable suggestion, and this result will be incorporated in the manuscript.

Gains are weaker for diverse generator (W1)

We agree that the gains are the largest for Complexa, but we believe the improvements for Genie3 and RFDiffusion3 are meaningful. Even though these generators already produce diverse binders, CGA yields 1.13× and 1.16× as many unique hits, respectively, at matched compute. This improvement is meaningful because CGA can be drop-in replacement to naive sampling (with zero GPU overhead and negligibly small CPU overhead), and also given the fact that widely-used reward-tilting methods often underperform naive sampling.

It is true that if every generated candidate belongs to a new cluster, CGA has no room to exploit and reduces to naive generation and evaluation. However, all three generators in our experiments exhibit structural redundancy, and this redundancy also persists in newer models like RFD4-Proteina [1] (RFD4-Proteina’s Supplementary Fig 16-17).

We also emphasize that CGA is one contribution within a broader framework we propose for understanding and forecasting unique hit scaling in binder design campaigns. As discussed in the Conclusion, diversity-guided generation could increase the number of discovered clusters (K), providing a complementary route to improvement. Even in the limit of highly diverse generation, selective evaluation could remain useful if additional quality information were available before expensive evaluation, for example, from a cheaper structure prediction model.

Finally, the benefit of allocation depends jointly on cluster discovery and cluster quality, rather than on uneven cluster sizes alone. Retire-on-hit avoids redundant evaluations, while least-failures-first redirects evaluation toward more promising unhit clusters. The latter is useful when failures provide strong evidence of low cluster quality, as characterized by the half-life of hmh_m in Fig. 5D.

[1] Biomolecular design through latent reasoning with RFD4-Proteina, 2026

Analysis of whether continuous scores helps (W2)

To clarify our motivation, we chose binary outcomes because the campaign objective is to discover clusters containing a candidate that satisfies all hit criteria. This allows the quality model to predict the event of interest directly; incorporating continuous scores requires relating their histories to the probability of jointly satisfying those criteria. We agree that continuous histories may contain useful information beyond failure counts, and will revise the manuscript to clarify both the rationale for our binary formulation and its limitations.

To assess the practical value of this additional information, we conducted a preliminary ablation on six RFDiffusion3 antigens. We replaced CGA’s least-failures-first evaluation rule with a minimum-min_interface_PAE-first rule, prioritizing the unhit cluster containing the candidate with the lowest min_interface_PAE. The resulting unique hit counts are:

AntigenCGAmin-min_interface_PAE-first
VEGFA3032
PD-L1248260
PD-1166152
Der f 76266
BBF143940
SC2RBD3043
Total575593

The new rule increased total unique hits from 575 to 593 (+3.1%), with improvements on five of six targets. The benefits were target-dependent: SC2RBD improved substantially, from 30 to 43 unique hits, while PD-1 decreased from 166 to 152. These preliminary results support the potential value of continuous evaluation histories. We will include this ablation in the revised manuscript and discuss more concrete path to incorporate continuous scores as an extension of the current quality model.

Reviewer_PVytOpenReview ↗

Thanks for the thorough response

Thank you for responding to all the points raised in the review. I appreciate the clarifications and the substantial experiments. These comparisons address my request and help isolate the contributions of the different components.

How were the four targets within each yield group selected? Also, were the CGA and naive reference results reuse the original runs and use the same reporting budget?

How much of the variation in preferred schedules across generators do you think can be explained by their generation/evaluation cost ratios? In particular, Complexa’s cheaper generation makes larger generation-to-evaluation ratios (e.g. GGGGE) less costly.

I remain uncertain about the benefit of the adaptive generation/evaluation decision against these simple baselines. GGGGE exceeds CGA for Complexa in all three yield-group aggregates, both GE and GGE are highly competitive with CGA for Genie3, and GGE is also competitive for RFD3, although CGA achieves stronger low and medium yield results. For a real design campaign, one could envision selecting a fixed schedule using pilot targets and applying it to subsequent campaigns, which seems like a plausible practical alternative. I recognize CGA’s advantage of avoiding schedule tuning and its stronger pooled performance in the low- and medium-yield groups.

Reviewer_SD17OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to maximize the number of unique computational hits in protein binder design under a fixed compute budget. It models cluster discovery and hit discovery within clusters, and proposes cluster-guided allocation (CGA) to decide when to generate candidates and which candidates to do in-silico evaluation. Experiments compare CGA with naive sampling and reward-tilting methods across 19 targets and three generators.

Critical strengths
  • The paper addresses a relevant practical problem. Generating more hits does not necessarily produce a more diverse library, and accounting for redundant evaluations is a reasonable way to improve compute efficiency.
  • The experiments cover multiple targets and generators, with comparisons at matched computational budgets
Critical weaknesses
  • The related work and baselines are poorly aligned with the main contribution. Most of the discussion concerns protein generation and reward tilting, while the proposed contribution is about resource allocation. The connection to multi-armed bandits is mentioned but insufficiently developed. Reward tilting is also largely orthogonal to allocation. A more relevant comparison would combine retire-on-hit and least-failures-first with a simple generation–evaluation schedule.
  • The formulation is unnecessarily difficult to follow. Section 3 is overly verbose. Although an objective is provided, the state, observations, assumptions, and decision process are spread across the text. For a paper centered on resource allocation, I would expect a clearer mathematical formulation connecting the objective to the proposed policy. Figure 2 is also difficult to interpret.
  • The statistical assumptions need stronger justification. The model assumes conditionally i.i.d. Bernoulli outcomes within a cluster and independent cluster qualities drawn from a shared distribution. Structural similarity alone does not establish these assumptions. The Pitman–Yor and zero-inflated Beta choices have reasonable motivations, but their effect on allocation performance and robustness to misspecification remain unclear.
  • The reliability of the estimated probabilities is insufficiently evaluated. Using binary hit outcomes simplifies modeling, but the allocation policy still depends on estimated probabilities. Agreement between predicted and observed aggregate yields does not establish that individual cluster probabilities are accurate.
  • The theoretical results have limited relevance to the full policy.
  • The methodological novelty and significance are insufficiently established. CGA combines intuitive rules -- retiring successful clusters and prioritizing clusters with fewer failures, with standard probabilistic models and a short-lookahead allocation heuristic. The paper does not clearly establish what new algorithmic insight this combination provides or whether its complexity is necessary for the reported gains.
Questions for authors
  • Can the authors compare CGA against retire-on-hit plus least-failures-first with a simple generation–evaluation schedule? This would help isolate the value of the fitted probability models and adaptive generation decision.
  • What evidence supports the conditional i.i.d. assumption within clusters and independence of cluster qualities? How sensitive are the results to violations of these assumptions and to the chosen Pitman–Yor and zero-inflated Beta models?
  • How accurate are the estimated cluster-level hit probabilities, and how do estimation errors affect allocation decisions? Can the authors evaluate this beyond aggregate campaign-yield predictions?
  • How does CGA relate to existing methods for sequential resource allocation and adaptive discovery? What is the methodological contribution beyond applying these ideas to binder design?
5 comments

Thank you for your thorough feedback on the presentation and theoretical results. We first address W1 and Q1 with expanded ablations comparing CGA against simpler generation-evaluation schedules, a comparison requested by many reviewers.

While we believe CGA is one contribution within our broader framework for understanding and predicting unique hit scaling in binder design campaigns, we agree that these ablations are important for assessing the practical value of its individual policy components, especially fitted probability models and adaptive generation decision.

We will follow up on your remaining concerns and questions shortly.


CGA vs retire-on-hit plus least-failures-first with a simple generation–evaluation schedule (W1, Q1)

We expanded the retire-on-hit-only ablation to all three generators and 12 targets, with four targets from each of the low-, medium-, and high-yield groups. We also added simpler, fixed generation-evaluation schedules, with different ratios. We now compare the following policies:

NameRetire-on-hitLeast-failures-firstGenerate vs Evaluate
NaiveNoNo1:1
Retire-on-hit onlyYesNo1:1
GE (new)YesYes1:1
GGE (new)YesYes2:1
GGGGE (new)YesYes4:1
CGA (ours)YesYesAdaptive decision, via proposed index

We uploaded the full per-target unique hit yield CSV and radar chart visualization to this anonymous link: https://anonymous.4open.science/r/rebuttal-scaling-0666/ablation_full_results.csv, https://anonymous.4open.science/r/rebuttal-scaling-0666/ablation_radar.pdf. Below, we summarize the results.

First, the expanded ablation in all 3 generators and 12 targets confirms that retire-on-hit component alone does not account for CGA’s gains. Relative to retire-on-hit only, CGA increases aggregate unique hit yield by 35.6%, 20.8%, and 3.8% in the low-, medium-, and high-yield groups, respectively.

Second, comparing CGA with GE, GGE, and GGGGE assesses the added value of CGA’s third component (using fitted probability models to adaptively choose between generation and evaluation) as requested by several reviewers. All three schedules are a strong baseline because it already incorporates our central idea of cluster-aware allocation and two of CGA’s three components: retire-on-hit and least-failures-first. The table below reports fold improvements over naive sampling, also visualized in the radar chart. The best policy for each yield group is bolded, and the second-best is underlined.

GroupGeneratorGEGGEGGGGECGA
LowComplexa1.33×1.71×2.54×2.25×
LowGenie 30.67×0.78×0.78×0.78×
LowRFdiffusion30.87×1.11×0.83×1.27×
LowAll three0.97×1.23×1.25‾\underline{1.25}x1.47×
MediumComplexa1.17×1.53×1.78×1.70×
MediumGenie 31.21×1.17×0.92×1.21×
MediumRFdiffusion31.04×1.15×1.15×1.31×
MediumAll three1.16×1.31‾\underline{1.31}×1.30×1.42×
HighComplexa1.42×1.61×1.82×1.57×
HighGenie 31.13×1.12×0.87×1.12×
HighRFdiffusion31.13×1.23×1.13×1.18×
HighAll three1.18×1.24×1.11×1.22‾\underline{1.22}x

The variation in fixed-schedule performance across groups justifies adaptive allocation: increasing the generation-evaluation ratio from 1:1 (GE) to 4:1 (GGGGE) improves performance for high-yield Complexa from 1.42× to 1.82× over naive, but reduces it for high-yield Genie 3 from 1.13× to 0.87×. Thus, a schedule that benefits one campaign can hurt another, highlighting the value of adapting this ratio to each campaign.

CGA delivers consistently strong performance across groups, achieving the highest aggregate improvement in the low- and medium-yield groups and remaining close to the best fixed schedule in the high-yield group.

Thanks again for the valuable suggestion, and this result will be incorporated in the manuscript.

Presentation (W1, W2)

Thank you for the feedback. We made these changes in the updated manuscript:

  • Related works: We agree that the connection to sequential allocation deserves a more explicit discussion. We originally put those discussions in Sec 3.3 and Appendix B.1; In Appendix B.1 we cited arm-acquiring and branching bandits, including Fershtman & Pavan (2021), whose search-experimentation tradeoff parallels our generation versus evaluation. We moved content in Sec 3.3 to Preliminary with the consolidated content.
  • Section 3: We revised Sec. 3.2 to present the state, observations, actions, costs, rewards, and objective together, with the relevant assumptions made explicit. The edited text are in blue. We kept the two underlying processes and their modeling assumptions in Sec. 3.1 to distinguish the statistical model from the allocation problem.

Significance/novelty of our work (W5-6, Q4)

Our central contribution is a cluster-level framework connecting structural cluster discovery, hit discovery within clusters, and computational cost to unique hit yield. For binder design, the key insight is that improving candidate-level hit rate does not necessarily improve the number of distinct hits obtained under a fixed compute budget: structural redundancy and evaluation cost also matter. Our framework makes these factors explicit and identifies compute allocation between generation and evaluation as a practical opportunity for improving campaign yield. CGA is one concrete policy developed within this framework, with empirical improvements over existing methods and ablations clarifying the contributions of its components.

We believe the novelty and significance of our work lie in bringing this perspective, framework, and policy to binder design and demonstrating their practical value. Our paper was submitted under the primary area of “applications to physical sciences”, and we would appreciate having its contributions considered in this applied context. The theoretical results provide supporting analysis under clearly stated assumptions and limitations. Our aim is to advance the understanding and practice of binder design, rather than to establish a general methodological advance in sequential resource allocation field.

Existing bandits methods cannot be applied directly to our practical setting. An important distinction from closely related work, such as Fershtman & Pavan (2021), is that generation can replenish an existing cluster’s finite inventory of unevaluated candidates. Because a cluster can be evaluated only when a candidate is available, generation changes the states and available actions of existing arms as well as introducing new ones. This coupling falls outside the independent arm structure underlying their optimality guarantees. We therefore develop CGA as a practical policy with a heuristic generation-evaluation decision and separate guarantees for retire-on-hit and least-failures-first under the assumptions stated in Appendix B. We explicitly acknowledge in the manuscript that these component-level guarantees do not establish optimality of the full policy.

We now made this distinction more clear in the Preliminary section.

Conditional i.i.d. assumption within clusters and independence of cluster qualities (W3, Q2)

Within-cluster conditional i.i.d. is motivated by independent sampling from generator rather than structural similarity. With a fixed generator and partition, candidates assigned to cluster are independent draws from the same conditional distribution.

Independence of cluster qualities is a separate simplifying assumption, abstracting away explicit relationships between clusters. We assess the practical adequacy of this assumption through predictive accuracy and campaign performance.

Estimation of cluster-level hit probabilities (W4, Q3)

We clarify that CGA does not fit a separate, unconstrained hit probability for every cluster. Instead, it uses the shared posterior predictive quantity hmh_m: the probability that the next candidate from an unhit cluster succeeds after mm previous failures. These fitted probabilities determine the comparison between generation and evaluation actions. The choice of the cluster to evaluate follows least-failures-first.

To evaluate the probabilities used by CGA beyond aggregate campaign-yield predictions, we fit the quality model using the first 8 GPU-hours and compare its predicted hmh_m with empirical next-hit frequencies observed between hours 8 and 22 in this figure*. The predictions capture the broad variation in next hit probabilities across generator-target pairs, with close agreement for fresh clusters in many campaigns. Agreement is less consistent after repeated failures, including underestimation in several Genie 3 campaigns. However, wide empirical confidence intervals due to low hit count limits the conclusion in such campaigns.

*The shaded band around the 8-hour fit represents a 95% confidence interval for the posterior mean of hmh_m. As in Fig. 3, the comparison is restricted to generator–target pairs with at least 20 observed hits. To reduce sampling noise, we group failure counts into bins of mm=0, 1, 2, 3-4, 5-9, 10-19, and 20+, plotting each bin at its mean failure count. Error bars on the empirical frequencies show 95% confidence intervals obtained by bootstrapping clusters.


Estimation errors can affect how compute is divided between generation and evaluation. Overestimating the value of repeatedly unsuccessful clusters relative to fresh clusters can favor further evaluation, while underestimating it can favor generation. The effect also depends on candidate availability and new cluster discovery probabilities.

To examine this, we tested a simplified quality model that uses a single empirical hit probability pp for all clusters, regardless of their failure counts. At each decision state, we update pp as the cumulative number of hits divided by the cumulative number of evaluated candidates. This variant retains both retire-on-hit and least-failures-first, allowing us to examine how quality estimates affect the generation-evaluation decision and resulting campaign performance. We ran this ablation with Complexa on four targets.

TargetNaiveCGAShared empirical p
Insulin168 (82.1%)247 (73.1%)202 (78.6%)
Derf2117 (87.9%)42 (66.9%)19 (87.2%)
TrkA109 (82.8%)155 (70.9%)154 (73.3%)
PD-L1117 (81.7%)227 (54.6%)186 (68.7%)

This model of cluster quality produce substantially different allocation decisions and thus campaign outcomes. While limited to four targets and one generator, these results provide preliminary evidence that modeling how hit probability change with failure counts improves the generation-evaluation tradeoff and, consequently, campaign yield.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.