acceptodds
Under review as a conference paper at ICLR 2027

BindGRPO: Training Protein Binder Generators to Bind

11 views

Abstract

Generative models of protein binders are typically trained by maximizing likelihood on natural complexes. Binding is considered primarily at inference: many candidates are sampled, scored by a structure predictor, and the highest-confidence ones are retained. Binding is therefore not part of the generator's training objective. We introduce BindGRPO to close this gap by post-training a target-conditioned binder generator using predicted binding as the reward. We formulate the denoising sampler as a Markov decision process and define Gaussian policy transitions between consecutive denoiser inputs with tractable likelihoods. The reward is AlphaFold3's interface confidence for the co-designed sequence. Training therefore does not require an experimentally determined target–binder complex. On the Cao benchmark, BindGRPO increases the in-silico success rate from to , a gain over the same generator before RL fine-tuning. The improvement persists under an independent AlphaFold2 evaluator and across most evaluation thresholds, suggesting that the gain is not specific to AlphaFold3. All code and data will be released.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein Design, Reinforcement Learning, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_qQsfOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BindGRPO post-trains a target-conditioned protein binder generator using AlphaFold3 interface confidence as a reward. It constructs conditional Gaussian transitions from BoltzGen’s EDM sampler and applies a binder-coordinate GRPO surrogate to fine-tune the denoiser. On the Cao benchmark, computational success increases from 24.4% to 36.7% after ProteinMPNN sequence reassignment, with improvements also observed for directly generated sequences and under an AlphaFold2 evaluator. The paper additionally examines threshold sensitivity, success-criterion components, and transfer to seven further targets, where the improvement is smaller and statistically inconclusive.

Critical strengths
  1. The main comparison convincingly isolates the benefit of post-training. BoltzGen and BindGRPO differ only in denoiser weights under the same evaluation protocol. The improvement after ProteinMPNN sequence reassignment supports better generated backbones, rather than merely more favorable decoded sequences.

  2. Validation goes beyond the optimized reward. AlphaFold2 corroboration, threshold sensitivity, paired per-target tests, and decomposition of the success criterion strengthen the evidence for improved computational design quality. The authors also report unfavorable results and explicitly acknowledge the absence of experimental binding validation.

Critical weaknesses
  1. The generator could improve by producing successful candidates across multiple structural families or by concentrating on a few high-scoring solutions. The reported success rates, evaluator checks, and threshold analyses do not distinguish these possibilities. Evidence about sequence and structural diversity among successful candidates would clarify whether post-training expands useful design options or mainly increases repeated sampling of similar solutions.

  2. Appendix D supports the combined training-pool filter through improved validation ipTM, but does not isolate the contribution of the reward-standard-deviation criterion from mean-reward filtering. Given GRPO’s within-group normalization, the role of this criterion remains unclear. Moreover, the reference-regularization ablation demonstrates that validation ipTM can diverge from benchmark success, limiting what validation reward alone establishes about the filter’s downstream benefit.

  3. The two-step AlphaFold3 reward approximation is validated using ProteinMPNN sequences on pretrained BoltzGen backbones, whereas RL uses co-designed sequences from an evolving generator. The reported ranking agreement therefore does not establish approximation fidelity on the samples actually driving optimization. This leaves uncertainty about whether improvements under the accelerated reward consistently reflect improvements under the default evaluator.

Questions for authors
  1. How does post-training affect sequence and structural diversity among successful candidates, and does the improvement correspond to more distinct successful designs?

  2. What contribution does the standard-deviation criterion make beyond mean-reward filtering, and does the filtering benefit also appear in downstream design success rather than only validation ipTM?

  3. Does the agreement between two-step and default AlphaFold3 evaluation remain high for co-designed sequences from later RL checkpoints, particularly those showing the largest reward improvements?

6 comments

We sincerely thank the reviewer for their thoughtful feedback and insightful suggestions. Based on your comments, we have conducted additional experiments and analyses, and re-evaluated the models across our benchmarks. We summarize the new results and address each point in detail below.

W1 — multiple structural families versus repeated sampling

We cluster the successful designs on the 11 Cao targets (1,100 designs per method, with 1100 structural proposals and 8800 proteinMPNN sequences) by sequence (50 % identity) and by binder backbone (4 Å Kabsch RMSD on index-aligned Cα atoms. ). For sequence cluster, we cluster the passing sequences and one successful design contributes anywhere from one to eight of them.

GeneratorAF3 success (%)Successful designsSequence clustersBackbone clusters
BindGRPO (ours)37.7415389211
BoltzGen (pretrained)26.3289280181
PXDesign15.2167167115
Proteina-Complexa13.4147145135
RFdiffusion39.9109109101
ODesign5.2575757

We further compute the sequence and structural diversity regardless of success or not for BoltzGen and BindGRPO.

PanelModelMean pairwise seq. identity (%)Mean pairwise backbone TMBackbone clusters / 100 designs
CaoBoltzGen12.90.33247.3
CaoBindGRPO13.80.35044.2
Held-out (50 targets)BoltzGen22.80.42650.7
Held-out (50 targets)BindGRPO22.10.43247.8

Findings:

  1. In absolute terms post-training yields more distinct successes. Against pretrained BoltzGen, BindGRPO has 44 % more passing designs (415 vs 289), 39 % more sequence clusters (389 vs 280) and 17 % more backbone clusters (211 vs 181), and it leads every baseline on both cluster counts.
  2. Sequence diversity is nearly unchanged and the binder structure distribution narrows slightly: Mean pairwise identity differs by +0.9 and −0.6 points. Backbone pairwise TM increases and Backbone clusters per 100 designs falls slightly.

In short: post-training buys more distinct successful designs in absolute terms and leaves sequence diversity intact, at the cost of a modest narrowing of the backbone distribution.

W2 — Isolating the spread criterion and validation selection

1. Why group normalization requires a spread filter. The training reward has its internal noise (unlike a reward in math problem) (R=Rtrue+εR = R_{\text{true}} + \varepsilon), the observed group variance is s2=strue2+σε2s^2 = s_{\text{true}}^2 + \sigma_{\varepsilon}^2. For targets whose designs have uniform true quality (strue≪σεs_{\text{true}} \ll \sigma_{\varepsilon}), dividing by ss in GRPO scales pure evaluator noise up to unit variance, turning noise into full-magnitude policy gradients. A mean filter (ipTM∈[0.2,0.8)\text{ipTM} \in [0.2, 0.8)) cannot catch low-spread targets anywhere inside that interval. Instead, our threshold (SD≥0.0784\text{SD} \ge 0.0784) is set directly above AF3's seed-to-seed reward noise floor (SD≈0.07\text{SD} \approx 0.07), retaining only targets with signal above evaluator noise.

2. Isolating mean vs. spread filters. While Appendix D showed the combined filter improves validation ipTM over no filtering, we further isolated each filter at 500 updates (1,000 targets):

Training pool filter (1,000 targets)Best val ipTMCao AF3 success (%, best-val ckpt)Cao AF3 success (%, update 500)
Mean + spread filter (Ours)0.47837.735.8
Spread filter only0.48534.535.6
Mean filter only0.46036.836.3

Combining both filters yields the strongest best-validation Cao success (37.7%), whereas spread-only achieves the highest validation ipTM (0.485) and mean-only matches at update 500.

3. Validation ipTM vs. three-gate success for checkpoint selection.

The three-gate criterion of Zambaldi et al. (2024) was specifically calibrated for the Cao benchmark protocol—61–65 residue miniproteins evaluated via any-of-eight ProteinMPNN sequence redesign. Applying those strict binary cutoffs directly to raw co-designed sequences on our diverse validation set yields an extremely sparse pass rate (∼3%\sim 3\%, or only ∼30\sim 30 passing designs per checkpoint).

To verify this, we scored 14 saved checkpoints (Run 3) on the validation split under both criteria:

Validation selection criterionSelected updateVal design success (%)*Cao AF3 success (%)
Highest validation ipTM (reported protocol)4403.437.4 (411/1,100)
Highest validation three-gate success2203.831.8 (350/1,100)

*Note: Validation success is per raw co-designed sequence, whereas Cao success is backbone-level any-of-eight with ProteinMPNN

Because the Cao-calibrated binary gates are overly stringent for un-redesigned PDB validation samples, they separate trained checkpoints (2.3–3.8%2.3\text{–}3.8\%) from pretrained BoltzGen (0.7%0.7\%) but are dominated by counting noise when ranking trained checkpoints. Under KL regularization (λKL=0.01\lambda_{\text{KL}} = 0.01), continuous validation ipTM avoids this threshold sparsity and reliably identifies the stronger checkpoint.

W3 — approximation fidelity on the samples that drive optimisation

We have expanded the paired comparison (2-step vs. 200-step AlphaFold3, keeping sequences, MSAs, and seeds fixed) to cover the generator's co-designed sequences across RL checkpoints (boltzgen, final, and the unregularized λKL=0\lambda_{\text{KL}} = 0 run), across both the Cao panel and a 100-target training subset:

Sequence Source & CheckpointDatasetPooled Spearman ρ\rhoWithin-Target Median ρ\rhoMean ipTM Offset (2-step − 200-step)Median Binder Cα\text{C}\alpha RMSD (A˚\text{\AA}, binder-only fit)
ProteinMPNN (boltzgen)Cao (1,100)0.9710.967−0.047-0.0470.50
Co-designed (boltzgen)*Cao (1,100)0.9600.931−0.057-0.0570.66
Co-designed (BindGRPO)Cao (1,100)0.9760.968−0.047-0.0470.46
Co-designed (unregularized, λKL=0\lambda_{\text{KL}}=0)Cao (1,100)0.9770.965−0.037-0.0370.39
Co-designed (boltzgen)*Train subset0.9430.932−0.033-0.0332.28
Co-designed (BindGRPO)Train subset0.9780.971−0.033-0.0330.88

Ranking agreement does not degrade under the cheaper reward: the within-target median ρ\rho on co-designed sequences rises from 0.931 before RL to 0.968 after, matching the 0.967 the paper reported for ProteinMPNN sequences, and from 0.932 to 0.971 on training targets. The two-step ipTM sits 0.03–0.06 below the full-step value in every row, a near-constant offset that leaves rankings intact. Re-folding the same sequences under a different AlphaFold3 seed gives a pooled ρ\rho of 0.95–0.96, so these values are at the evaluator's own noise ceiling.

In short: the two-step and 200-step settings do not disagree in any way that affects training. They rank co-designed sequences the same, before and after RL alike; what disagreement remains is a near-constant offset of about 0.05 ipTM, which cancels in the group-relative advantage; and the residual is no larger than full-step AlphaFold3's disagreement with itself across seeds.

Reviewer_vbTcOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The BindGRPO work proposed a framework to finetune, using reinforcement learning, the BoltzGen diffusion module towards better folding model metrics. The authors formulated the EDM sampler of BoltzGen as a policy, and carefully curated a training target group that provides enough training signal during RL. The model after RL shows improvement over the base model in terms of in-silico success.

Critical strengths
  1. Formulation of the BoltzGen EDM sampler as a policy makes it possible to apply a Flow-GRPO based algorithm.
  2. Concrete improvement over the base BoltzGen model checkpoint.
  3. The paper provides strategy to curate a task specific RL target set.
Critical weaknesses
  1. The paper did not show address the diversity-success trade-off after RL. See questions for detail.
  2. The exploration of alternative reward formulation is limited.

See questions for other matters.

Questions for authors
  1. Why do the authors only experiment with finetuning with the AF3 ipTM reward, instead of a combination of several confidence metrics that gate the in-silico success?

  2. If the re-designed success rate is to be reported as the main result, why not include the sequence re-design phase also in the reward calculation?

  3. I think the RL target selection proposed by the authors is a nice strategy. One question I have is that how the final 1,000 training targets are selected from the 1,489 eligible candidates (as shown in Figure 5a)? Please elaborate.

  4. My most important question is: RL tends to sharpen the distribution and may reduce the generated sample diversity as a result. In the paper the authors reported design successes, but not unique success in terms of structure and sequence. Can the authors report unique successes generated by each method?

  5. From my understanding, Proteina-Complexa can design against multimer targets. Could the authors clarify the reason for the exclusion of those multi-chain targets in Table 1 for Proteina-Complexa?

  6. The BindGRPO and the baselines are evaluated for generating binder of a very specific length (61-65 residues), I'm wondering how the comparison would change if we increase the range to cover smaller proteins like peptides with <20 residues or larger binders with ~100 residues.

  7. Reference regularization with KL penalty: the authors report that zero KL penalty leads to better validation reward but lower in-silico success (Appendix D). If the model learns to generate samples with higher ipTM, what's the gating filter that leads to reduced success?

9 comments

We sincerely thank the reviewer for their thoughtful feedback and insightful suggestions. Based on your comments, we have conducted additional experiments and analyses, and re-evaluated the models across our benchmarks. We summarize the new results and address each point in detail below.

W1 — diversity–success trade-off

Answered in Q4.


W2 — alternative reward formulations

Answered in Q1.


Q1 — reward formulation: why a single confidence metric

We optimise the single metric AlphaProteo found most predictive of experimental binding (their Fig. S1B), which also carries the dominant weight in AlphaFold3's own ranking score (0.8 ipTM+0.2 pTM0.8\,\text{ipTM} + 0.2\,\text{pTM}). Optimizing a single, well-validated metric keeps the post-training objective simple and clean. Keeping the training reward narrower than the evaluation filter allows us to detect over-optimization.

Nevertheless, our framework is agnostic to the specific reward function. Starting from the same pretrained checkpoint we trained two further models. The three-gate reward is the one the reviewer proposes; We adopt a smooth form of the three evaluation thresholds themselves: interface PAE < 1.5, binder pTM > 0.8 and complex RMSD < 2.5 Å , scored by the same two-step AlphaFold3 call and over the same 500 updates. The geometry reward is a deliberately mis-specified control that needs no structure prediction at all: the strand fraction minus the helix fraction of the binder's Cα trace, over 200 updates.

Training rewardCao AF3 success (%)Cao AF2-IG success (%)Held-out reward, start → end
BoltzGen (pretrained)26.329.1—
Geometry reward (strand minus helix)——−0.31→+0.71-0.31 \to +0.71
Three-gate reward38.541.20.006→0.0200.006 \to 0.020
AF3 ipTM reward (BindGRPO)37.743.00.31→0.480.31 \to 0.48

The geometry reward was optimised as intended: its own held-out value rose from −0.31-0.31 to +0.71+0.71 over 200 updates and the policy followed the reward it was given.

The three-gate reward matches ipTM and does not beat it. It trains stably with held-out reward 0.006→0.0200.006 \to 0.020, held-out interface PAE of its co-designed samples 11.5→8.611.5 \to 8.6 Å, and reaches 38.5 % at its best-validation-iptm checkpoint and 36.7 % at update 500, against BindGRPO's 37.7 % on the same panel. Under AlphaFold2-IG it is slightly behind (41.2 % against 43.0 %).

The three-gate run does change which thresholds its passing sequences clear: 45.6 % clear binder pTM against BindGRPO's 38.0 %, 26.5 % clear interface PAE against 30.0 %. This shows a trade-off between optimizing different metrics; careful adjustment of the reward-term weights may yield a better model.

Q2 — why the re-design phase is not in the reward

We trained this variant to the full 500-update budget, with one ProteinMPNN redesign per backbone at the same AlphaFold3 cost as co-design. It is not better: 36.1 % against 37.7 %.

The reason is also measurable without training. Nesting a stochastic redesign step inside the rollout makes the observed reward Rk=Rtrue(xk)+ξkR_k = R_{\text{true}}(x_k) + \xi_k, and GRPO divides by the group standard deviation, so the per-backbone sampling noise enters the denominator of the advantage:

spread2=spreadtrue2+σMPNN2/N\text{spread}^2 = \text{spread}_{\text{true}}^2 + \sigma^2_{\text{MPNN}}/N

We measure the noise introduced by ProteinMPNN over 100 training targets × 10 backbones (1000 designs), with eight ProteinMPNN sequences per backbone. σdesign\sigma_{\text{design}} is the spread of true design quality. The SNR is signal over total group spread, which includes AlphaFold3's own evaluation noise (SD ≈ 0.07, measured by re-folding fixed sequences under a different seed), so it stays below 1 even for co-design.

Reward sequence sourceσdesign\sigma_{\text{design}}σMPNN\sigma_{\text{MPNN}}Advantage SNRCao success (%)
In-loop ProteinMPNN redesign0.1480.1410.6936.1
Co-designed sequence (ours)0.207—0.9537.7

Across 1,000 designs, redesign noise on a fixed backbone (σMPNN=0.141\sigma_{\text{MPNN}} = 0.141) is as large as the design spread (σdesign=0.148\sigma_{\text{design}} = 0.148) within each group it has to be distinguished from. Recovering co-design's signal-to-noise would take 1.9× to 8.2× more AlphaFold3 folds.

It is also unnecessary: BoltzGen co-designs sequence and structure jointly, and the backbone improvements transfer to redesigned sequences, raising per-sequence ProteinMPNN success on the same backbones from 12.8 % to 18.7 %.

Q3 — how the 1,000 training targets are selected

The 1,000 training targets are chosen by matching to PDB targets. We draw 1,000 reference targets at random from the PDB targets that pass the same reward filter (mean ipTM in [0.2, 0.8), within-group SD ≥ 0.0784), and pair each with the nearest unused eligible Bindome target in the plane of mean ipTM and within-group SD. Fixing the training pool to exactly 1,000 targets was a deliberate experimental control rather than a limitation of the remaining 489 eligible targets. When comparing different target-selection strategies during development, capping every strategy at 1,000 targets ensured a fair comparison under an identical training-target budget. We retained this 1,000-target pool for the main model, and we also trained a model on the full pool of all 1,489 eligible targets for comparison:

Training poolTargetsVal ipTMCao AF3 success (%)
BoltzGen (pretrained)—0.30826.3
BindGRPO (reported pool)1,0000.47837.7
BindGRPO (full eligible pool)1,4890.46136.1
Reviewer_ahUgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BindGRPO, a reinforcement-learning framework for post-training a target-conditioned protein binder generator using AlphaFold3 interface confidence as the reward. Starting from BoltzGen, the authors construct tractable Gaussian transitions by combining each deterministic denoising update with the subsequent stochastic augmentation and noise injection. They optimize a GRPO-style surrogate using coordinate-averaged likelihood ratios over binder atoms and reference-policy regularization. Training rewards are computed directly from co-designed sequences using a reduced-step AlphaFold3 configuration.

On the eleven-target Cao benchmark, BindGRPO improves computational success from 24.4% to 36.7% after ProteinMPNN sequence reassignment. Improvements also appear under AlphaFold2 initial-guess evaluation and when evaluating co-designed sequences directly. Additional experiments examine seven further targets, evaluation thresholds, reward approximation, and optimization settings. The results support the usefulness of reward-based post-training, although the specific benefits of the proposed optimization approach and the breadth of generalization remain incompletely established.

Critical strengths

A practical approach to post-training a complete binder generator. The paper optimizes a model that generates complete binder sequences and structures, rather than restricting optimization to sequences or local structural regions. The construction of Gaussian transitions provides a workable way to apply likelihood-based optimization to the existing EDM sampler. The authors also explicitly acknowledge that coordinate averaging, omitted target-coordinate terms, and stopped alignment gradients yield a training surrogate rather than the full sampler policy gradient.

Several complementary checks support the main improvement. The comparison between pretrained BoltzGen and BindGRPO holds the generation and evaluation pipeline fixed. The Cao benchmark improvement is reproduced under AlphaFold2 initial-guess evaluation, from 28.5% to 42.0%, and persists across most tested thresholds. Evaluating both native sequences and ProteinMPNN-reassigned sequences helps distinguish improvements in co-design from improvements in the usefulness of generated backbones. The use of target-level paired tests and the explicit discussion of dependence among sequences from the same backbone are also strengths.

Critical weaknesses

The experiments establish a benefit from post-training, but do not isolate the value of the proposed GRPO approach. The principal comparison is against the pretrained generator, while the external baselines are different generative models. There is no matched comparison with simpler post-training alternatives, such as reward-weighted regression or fine-tuning on high-reward generated samples, using the same starting checkpoint, target pool, and reward budget. Appendix D provides useful optimization ablations, but does not test the main policy-surrogate choices, including binder-only likelihood terms and coordinate averaging. Thus, the evidence does not establish which elements of BindGRPO are necessary for the observed gains. Reporting training cost and the number of distinct successful sequence or structural clusters would also help determine whether the higher pass rate provides a practical benefit without substantially reducing diversity.

The evidence for robust improvement beyond the optimized confidence proxy remains limited. AlphaFold2 evaluation is a useful additional check, but it starts from the generated complex and therefore tests a different, partially constrained prediction task rather than independent reconstruction from sequence. More importantly, Appendix D demonstrates a concrete mismatch between reward and downstream success: removing KL regularization increases validation ipTM to 0.5765 while Cao success falls to 22.0%. The reduced-step reward is validated on ProteinMPNN sequences from pretrained BoltzGen backbones, rather than on the evolving co-designed sequences used during RL, leaving its reliability under optimization uncertain. Generalization is also uneven: on the additional seven-target panel, success increases only from 21.1% to 23.9% with p = 0.313, and most of the gain comes from one target. Native co-design success improves on Cao, but is not significant at the target level either (p = 0.063). These results support a useful improvement on the primary computational benchmark, but provide weaker evidence for broadly transferable binder-design gains. The authors appropriately acknowledge the absence of experimental binding validation.

Questions for authors

Can you compare BindGRPO with at least one simpler post-training baseline using the same initial model, training targets, and reward-evaluation budget? Which policy-surrogate choices materially affect downstream success?

Does the two-step AlphaFold3 reward preserve rankings relative to full-step evaluation for co-designed sequences sampled during and after RL, including the unregularized run? This would help distinguish reward-approximation exploitation from other causes of the reward–success mismatch.

How were the final checkpoint and KL coefficient selected? Were Cao benchmark results consulted during these choices, and are the reported gains reproducible across independent training runs?

How does post-training affect sequence diversity, backbone diversity, and the number of distinct successful designs? Please also report the computational cost of target screening and RL training so that the practical trade-off can be assessed.

7 comments

We sincerely thank the reviewer for their thoughtful feedback and insightful suggestions. Based on your comments, we have conducted additional experiments and analyses, and re-evaluated the models across our benchmarks. We summarize the new results and address each point in detail below.

W1 — matched post-training baselines and policy-surrogate choices

We ran both Policy-surrogate ablations and Matched simpler post-training baselines from the same initial checkpoint, the same 1,000 training targets and the same reward budget.

VariantLikelihood scopeAggregationVal ipTMCao AF3 success (%)
BoltzGen (pretrained)——0.30826.3
Filtered SFT (top-reward samples)———31.3
Reward-weighted regression (soft weights on the same top-reward samples)———33.4
Reward-weighted regression (soft weights on all samples)———27.5
BindGRPO (binder-only, summed)Binder onlySum0.57927.1
BindGRPO (whole-complex, averaged)Target + BinderMean0.41833.2
BindGRPO (ours)Binder onlyMean0.47837.7

Simpler baselines. Soft reward weighting on the same top-reward samples reaches 33.4 %, not distinguishable from filtered SFT (31.3 ); on all samples it stays at the pretrained level (27.5 %). BindGRPO is 4.4 points above the better regression variant and 6.5 above filtered SFT. Each BindGRPO row is its best-validation checkpoint; the Filtered SFT and regression rows are final checkpoints.

Binder-only scope. BoltzGen denoises the whole complex and takes which tokens are designed and the structure & sequence of target protein as input. So there is nearly nothing to learn on the target protein. Also, the binder is a minority of the complex's atoms, so a whole-complex average spends most of the gradient outside the group being designed and outside what the reward scores. It costs 4.5 points of design success (33.2 % vs 37.7 %).

Coordinate averaging. Summing the binder log-density scales the policy term by 3Nbinder≈3.8×1033N_{\text{binder}} \approx 3.8\times10^{3} while leaving the KL term unchanged, cutting the effective λKL\lambda_{\text{KL}} to ≈2.6×10−6\approx 2.6\times10^{-6}. The run then reproduces the unregularized failure mode of C3: top validation ipTM, pretrained-level success. Averaging is what keeps the two terms on one scale. Further, coordinate averaging keeps the log-density terms for binders with different lengths at the same scale.

We do not claim GRPO is superior to other post-training objectives in general. For the architecture introduced by AlphaFold3 and adopted by BoltzGen, a frozen heavy conditioner followed by a lightweight diffuser with tractable Gaussian transitions, GRPO is a natural choice: the diffusion trajectories conditioned on one conditioner pass form a group.


W2 — training cost and distinct successful designs

We present both the computational cost analysis and the diversity results directly below.

1. Computational cost:

StageConfigurationCompute Cost (H200 GPU-hours)Relative to cevalc_{\text{eval}}
Target Screening3,792 targets × 40 designs (2-step AF3)≈ 298†≈ 22,000 ×ceval\times c_{\text{eval}}
RL Training (CtrainC_{\text{train}})500 updates × 256 designs (40-step gen + 2-step AF3)365≈ 27,000 ×ceval\times c_{\text{eval}}
Benchmark Eval (cevalc_{\text{eval}})Per design (500-step gen + 8×MPNN + 8×200-step AF3)0.01351.0×1.0\times

GPU-hours are GPUs × wall-clock. cevalc_{\text{eval}} measured on Cao (1,100 designs, 8 GPUs, 1.86 h); RL training on 8 GPUs (45.6 h), including held-out evaluations. † Screening was not timed; this applies the training run's per-design rates to the 151,680 designs scanned.

2. Unique successful designs on the Cao benchmark. We cluster the successful designs on the 11 Cao targets (1,100 designs per method, with 1100 structural proposals and 8800 proteinMPNN sequences) by sequence (50 % identity) and by binder backbone (4 Å Kabsch RMSD on index-aligned Cα atoms. ). For sequence cluster, we cluster the passing sequences and one successful design contributes anywhere from one to eight of them.

GeneratorAF3 success (%)Successful designsSequence clustersBackbone clusters
BindGRPO (ours)37.7415389211
BoltzGen (pretrained)26.3289280181
PXDesign15.2167167115
Proteina-Complexa13.4147145135
RFdiffusion39.9109109101
ODesign5.2575757

We further compute the sequence and structural diversity regardless of success or not for BoltzGen and BindGRPO.

PanelModelMean pairwise seq. identity (%)Mean pairwise backbone TMBackbone clusters / 100 designs
CaoBoltzGen12.90.33247.3
CaoBindGRPO13.80.35044.2
Held-out (50 targets)BoltzGen22.80.42650.7
Held-out (50 targets)BindGRPO22.10.43247.8

Findings:

  1. In absolute terms post-training yields more distinct successes. Against pretrained BoltzGen, BindGRPO has 44 % more passing designs (415 vs 289), 39 % more sequence clusters (389 vs 280) and 17 % more backbone clusters (211 vs 181), and it leads every baseline on both cluster counts.
  2. Sequence diversity is nearly unchanged and the binder structure distribution narrows slightly: Mean pairwise identity differs by +0.9 and −0.6 points. Backbone pairwise TM increases and Backbone clusters per 100 designs falls slightly.

In short: post-training buys more distinct successful designs in absolute terms and leaves sequence diversity intact, at the cost of a modest narrowing of the backbone distribution.

W3 — evidence beyond the optimised confidence proxy

We agree that AF2 initial-guess reuses the generated complex and so tests preservation rather than reconstruction. We therefore added two predictors that work from sequence alone, with independently trained weights and BindGRPO shows improvement under both predictors.

GeneratorAlphaFold3 (%)AlphaFold2-IG (%)Protenix v1 (%)Protenix v2 (%)
BoltzGen (pretrained)26.329.152.746.6
BindGRPO (ours)37.743.059.557.5

On the reward–success mismatch the reviewer identifies, we agree it is a real limitation of confidence as a reward, and we now report which criteria break. We decompose the three evaluation criteria across all 8,800 sequences on the Cao panel for the unregularized (λKL=0\lambda_{\text{KL}} = 0) checkpoint alongside Table 3:

Model / SettingVal ipTMPer-Seq Pass: min⁡iPAE<1.5\min \text{iPAE} < 1.5 (%)Per-Seq Pass: Binder pTM>0.8\text{pTM} > 0.8 (%)Per-Seq Pass: Complex RMSD<2.5 A˚\text{RMSD} < 2.5\text{ \AA} (%)Overall Cao Design Success (%)
BoltzGen (pretrained)0.30819.3%29.5%43.2%26.3%
BindGRPO (λKL=0.01\lambda_{\text{KL}} = 0.01)0.47830.0%38.0%52.4%37.7%
Unregularized (λKL=0\lambda_{\text{KL}} = 0)0.59433.1%17.4%53.4%26.6%

Removing the KL trust region allows the policy to exploit the ipTM reward while degrading the unrewarded structural self-consistency criteria (binder pTM and complex RMSD). This directly motivates keeping the KL regularization: with λKL=0.01\lambda_{\text{KL}} = 0.01, all three components improve substantially over pretrained BoltzGen. This is also why the training reward is deliberately narrower than the evaluation conjunction. Folding the gates into the reward would have hidden precisely this divergence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.