acceptodds
Under review as a conference paper at ICLR 2027

Online Search-Augmented Reinforcement Post-Training for All-Atom Protein Design

11 views

Abstract

Improving pretrained all-atom protein generators toward downstream biophysical objectives remains challenging but is underexplored. Reinforcement-based post-training can directly adapt the generator, but trajectory-level optimization over hundreds of denoising steps is expensive. Inference-time search can instead improve candidate quality through additional sampling and selection, but leaves the generator itself unchanged and must be repeated whenever new candidates are generated. We introduce Search-Augmented Reinforcement Post-Training (SARP), which combines trajectory-free endpoint-based reinforcement with temporary supervision on high-reward endpoints discovered by reward-guided search. Across small-molecule binder and enzyme-design tasks, SARP improves design quality under standard sampling of the partially latent protein all-atom generator. In small-molecule binder design, SARP decreases binding energy by up to 14.6 REU and RF3 min-iPAE by up to 2.2 Å relative to the pretrained model. Across five enzyme targets, SARP produces 42.2 unique successes per 100 designs, compared with 11.2 for the pretrained model and 35.6 for the RL-only method DGPO. These gains persist under independent AlphaFold3 cross-evaluation, suggesting that the improvements transfer across structure predictors rather than arising solely from optimization of RF3-specific signals. Gradient analysis further shows that search supervision increasingly conflicts with reinforcement over training and that the backbone and latent flows experience different optimization pressures, suggesting that adaptive control of supervision and regularization may be beneficial. Overall, SARP provides a practical post-training framework for protein generative models that enables fast standard-sampling generation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
protein design, protein generation, post training, reinforcement learning, flow matching

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_GzJPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SARP for a partially latent all-atom protein generator. SARP combines endpoint-based DGPO with auxiliary flow-matching supervision on high-reward endpoints obtained through beam search. The reinforcement update operates on freshly re-noised generated endpoints rather than stored sampling trajectories, while searched endpoints are used as separate supervised targets. The method also uses moving-reference regularization and controls the duration of search supervision. Both the explicit Cα backbone and non-Cα latent flows are updated. The empirical evaluation covers four small-molecule binder conditions and five independently optimized enzyme-design targets. Comparisons include trajectory-based GRPO, DGPO, search-only supervised fine-tuning, rollout-derived endpoint supervision, and inference-time search. The paper reports improvements in computational design quality and structurally distinct successful designs under standard sampling, provides evaluation-only AlphaFold3 cross-checks, and analyzes how supervision duration and search strength affect optimization and diversity.

Critical strengths

The method improves the yield of structurally distinct successful designs, with meaningful cross-evaluator support. The enzyme results are stronger than an improvement in average reward alone. Under RF3 evaluation, SARP obtains an average pass rate of 69.4% and 42.2 unique passing structural clusters per 100 designs, compared with 54.4% and 35.6 for DGPO; the pretrained model yields 11.2 unique passing clusters. This makes the improvement relevant to generating a portfolio of candidate designs rather than repeatedly producing similar high-scoring outputs. The ablations identify a consequential quality–diversity distinction between supervision sources. The author account for distributed supervision frequency and the number of accepted endpoints, and distinguish this source-control experiment from an independently optimized competing method. In enzyme design, rollout-derived supervision achieves comparable or slightly better average geometric metrics, but yields 21.0 unique passing clusters versus 42.2 for search-derived supervision. Thus, the empirical value of search is not presented solely as stronger fitting of local geometry; it also concerns which successful structures the generator continues to sample.

Critical weaknesses
  1. The practical advantage of search over cheaper endpoint supervision is not yet established under a matched total resource budget.This makes the additional cost of search important to the overall assessment. Moreover, the manuscript explicitly states that SARP-Rollout inherits SARP’s training horizons rather than being independently tuned, so this control does not establish the strongest achievable performance of the cheaper alternative. The doubled-group DGPO experiment is useful, but it does not establish equivalence in total generator computation or reward-evaluator calls. Appendix J reports relative inference-time search overhead, whereas the enzyme training budgets are explicitly not compute-matched. A complete accounting of post-training, search, and evaluation costs—and the number of subsequent designs needed to amortize those costs—is missing. This matters especially because enzyme models are optimized independently for each target. The current evidence supports improved standard-sampling quality, but not yet a general claim of better end-to-end efficiency than well-tuned rollout distillation or inference-time search.

  2. The robustness of the incremental gains and the reproducibility of model selection are insufficiently quantified. The principal comparisons report point estimates without specifying independent post-training seed counts or providing run-level uncertainty. Generating 100 enzyme designs from a checkpoint measures sampling outcomes, but does not quantify variability across training runs. Checkpoint selection and supervision termination are described through qualitative conditions such as a “steep drop” in diversity. I appreciate that the final evaluation set is explicitly excluded from checkpoint selection; my concern is not demonstrated test-set leakage, but whether another implementation or training run would make the same stopping decisions. Quantitative stopping rules and repeated-run results would materially strengthen the claim that search reliably improves the quality–diversity trade-off.

Questions for authors
  1. Resource accounting and the search-specific benefit. What are the total generator evaluations, reward-evaluator calls, GPU-hours, and post-training costs for SARP, DGPO, and SARP-Rollout? For the target-specific enzyme setting, approximately how many subsequent designs are required before SARP becomes cheaper than using the pretrained model with inference-time search?
  2. Training variability and stopping decisions. How many independent post-training runs underlie the main tables? Please specify the numerical rule, smoothing window, and any patience criterion used to disable supervision and select checkpoints. Does the advantage in unique passing clusters persist across training seeds and when SARP-Rollout receives its own training-diagnostic-based stopping schedule, rather than inheriting SARP’s horizon?
  3. Enzyme geometry definitions . Appendix E.3 defines the backbone reward using the RMSD of the generated catalytic-motif backbone relative to its target geometry, whereas Section 4.2 describes success using RF3-refolded backbone self-consistency. Please specify the structures, atom sets, and alignment procedures used for the backbone reward, the “BB RMSD” column, and the backbone success criterion. Are these distinct quantities, or is the wording inconsistent? This distinction is necessary to interpret what is optimized versus independently evaluated.
3 comments
AuthorsOpenReview ↗

Response to Reviewer GzJP – Weakness 1 and Q1: Resource Accounting and Search-Specific Benefit

We appreciate the reviewer's request for detailed computational accounting and more evidence for search's benefit. We report generator evaluations (NFEs), reward-oracle calls, training GPU-hours, and target-specific cost amortization. We have also updated the manuscript accordingly.

1. Computational cost.

With 32 rollouts and 400 denoising steps, DGPO and SARP-Rollout require 12,800 NFEs and 32 reward-oracle calls per rank-step. SARP incurs additional computation from reward-guided beam search, as quantified below. NFEs / rank-step denotes the number of generator evaluations per training step, and oracle calls counts reward evaluations. The corresponding multipliers indicate computational overhead relative to DGPO. Detailed NFE and reward-oracle accounting is provided in Appendix K.

MethodNFEs / rank-stepNFE multiplierOracle callsOracle multiplier
DGPO / SARP-Rollout12,8001.00×321.00×
SARP (W=2)20,8001.625×662.06×
SARP (W=8)44,8003.50×1685.25×

Training GPU-hours per optimization step:

TaskDGPOSARP-RolloutSARP (W=2)SARP (W=8)
Enzyme3.122.604.1010.04
Small-molecule binder3.342.873.657.44

Total training GPU-hours, including subsequent RL-only optimization:

TargetGRPODGPOSARP-RolloutSARP
M0375730.4289.0212.0240.0
M0732819.5346.0198.0275.0
M0552516.5377.8223.0513.0
M0904979.0313.0378.0485.0
M0157921.1580.0247.0528.0
Small-molecule binder776.8701.6574.4718.2 (W=2)

Per-step SARP costs apply to search-supervised steps. Full-supervision small-molecule SARP costs 718.2 GPU-h (W=2) or 1,513.5 GPU-h (W=8) over 200 steps.

2. Search-specific benefit and structural novelty.

SARP-Rollout replaces search-derived supervision with reward-selected on-policy endpoints under the same schedule. Across five enzyme targets, SARP achieves 42.2 unique passing clusters per 100 designs versus 21.0 for SARP-Rollout.

Besides, we jointly cluster accepted search candidates and concurrent rollouts using Foldseek (TM-score ≥ 0.5) to prove that search can find more novel passing structures. We define accepted novelty as:

Novelty=∣Caccepted∖Crollout∣∣Caccepted∣,\mathrm{Novelty}=\frac{|C_{\mathrm{accepted}}\setminus C_{\mathrm{rollout}}|}{|C_{\mathrm{accepted}}|},

where CacceptedC_{\mathrm{accepted}} and CrolloutC_{\mathrm{rollout}} denote structural clusters of accepted search candidates and concurrent ordinary rollouts, respectively.

TargetConfigurationAccepted noveltyAcceptance rate
M0375W=233.3% (32/96)37.5%
M0552W=878.4% (69/88)82.0%
M0732W=828.4% (27/95)83.6%
M0904W=820.0% (7/35)75.0%
Small-molecule binderW=245.1% (185/410)46.2%

Search discovers clusters absent from ordinary rollouts (47.2% for enzymes at W=8; 45.1% for small-molecule binders), providing structurally novel supervision.

3. Target-specific training–inference amortization.

On NVIDIA L40S GPUs, inference-time search requires 10.84× more GPU-hours per design than standard sampling (0.06374 vs. 0.00588 GPU-h/design), consistent with 4,000 versus 400 NFEs (10×). Accounting for pass rates, SARP requires 8.9–16.3× fewer marginal inference GPU-hours per passing design than pretrained inference-time search.

Including each checkpoint's full target-specific post-training cost:

TargetTrain GPU-hSARP GPU-h/passBase+search GPU-h/passBreak-even passes
M03752400.02450.3984642
M07322750.006610.08293,605
M05525130.008170.07997,154
M09044850.006680.07457,156
M01575280.007950.07058,440

GPU-h/pass excludes upfront training and is only for inference evaluation. Break-even calculations include full SARP training costs.

SARP amortizes training after 642–8,440 passing designs, or approximately 4,100–9,100 generated candidates at equal generation volumes. These scales are practical: previous studies generated 10,000–50,000 candidates for Ras isoform-selective binders and approximately 10,000 candidates for intrinsically disordered protein binders.

Full resource accounting, inference benchmarks, and amortization analyses are provided in Appendix K.

AuthorsOpenReview ↗

Response to Reviewer GzJP – Weakness 2 and Q2: Training Variability and Stopping Decisions

We appreciate the reviewer's concerns regarding training variability and stopping decisions. We clarify the training protocol and provide additional seed-sensitivity and training-dynamics analyses.

1. Training schedules and stopping criteria.

We clarify that search-supervision duration is an explicit training hyperparameter rather than an adaptive stopping criterion. No numerical diversity threshold, smoothing window, or patience criterion is used. The use of temporary supervision is supported by our empirical and optimization analyses. Our gradient analysis (Section 4.4 and Appendix Q) shows that Search-SFT becomes increasingly misaligned with DGPO during training, while our supervision-duration ablation (Appendix R, Table 23) demonstrates that prolonged search supervision can adversely affect structural diversity. Consistent with this concern, rollout diversity decreases by approximately 25% during the first 20 steps on M0732 and 16% on M0375, motivating us to turn off search supervision at step 20 rather than relying on a universal diversity threshold. Given the computational scale of all-atom protein post-training, we have not exhaustively optimized the turn-off step for each target. Nevertheless, the matched schedules and supporting analyses provide a controlled basis for evaluating the contribution of search-derived supervision.

2. Training variability across random seeds.

Due to the substantial computational cost, the main results are based on a single independent training run per method and target. Nevertheless, we evaluate SARP across multiple distinct design tasks, including small-molecule binder, enzyme, and DNA-binding protein design, and observe consistent improvements across these settings. Besides, to directly assess training-seed sensitivity, we additionally train SARP on enzyme target M0552 using a second random seed.

MethodStepsCat. match ↑Cat. RMSD ↓BB RMSD ↓Lig. clash ↓Pass ↑Unique pass ↑
SARP (seed 42)90(30)1.0001.360.570.003%72%54
SARP (seed 143)90(30)1.0001.410.630.000%76%51

Both seeds achieve comparable pass rates (72% vs. 76%) and unique passing clusters (54 vs. 51), outperforming DGPO on M0552 (60% pass rate, 40 unique clusters). This provides preliminary evidence of robustness to training seeds.

3. SARP-Rollout stopping schedules and diversity dynamics.

SARP-Rollout uses the same training horizon and supervision schedule as SARP to isolate the effect of supervision source. We acknowledge that an independently optimized stopping schedule could affect its performance and have not conducted a comprehensive schedule sweep due to the long training time and cost. However, additional analyses suggest that checkpoint selection alone does not explain the diversity gap.

(a) Quality–diversity dynamics. On M0375, both methods improve backbone RMSD at similar rates, but exhibit different diversity trends:

MethodPearson rCluster slope / stepBB RMSD slope / step
SARP−0.617 (n=27)+0.042−0.0018
SARP-Rollout+0.677 (n=32)−0.094−0.0020

SARP maintains or increases structural diversity as backbone RMSD improves, whereas SARP-Rollout tends to lose diversity.

(b) Diversity across training checkpoints. We examine training-rollout diversity throughout the observed post-supervision trajectories:

TargetSARP meanSARP worstRollout meanRollout bestUnique-pass gap
M090420.7014.752.607.25+51
M037521.3715.627.5119.12+10
M055223.9517.3820.6823.38+23
M015721.5315.1225.5528.620

On M0904, even SARP-Rollout's best observed diversity (7.25 clusters) is below SARP's worst (14.75), consistent with the evaluation-time unique-pass gap (52 vs. 1). On M0375, SARP maintains higher diversity at all 22 matched logged steps. The trend is weaker on M0552 and absent on M0157, indicating target-dependent benefits.

(c) Search-derived structural novelty. Our additional Foldseek analysis shows that 47.2% of accepted search clusters across three enzyme targets (W=8) are absent from concurrent ordinary rollout groups, with 45.1% novelty for small-molecule binders (W=2). This provides a complementary explanation for SARP's diversity advantage beyond selecting existing high-reward rollouts.

Overall, the additional seed experiment supports reproducibility on M0552, while the training-dynamics and structural-novelty analyses indicate that search-derived supervision provides diversity benefits beyond checkpoint selection. Independently optimized SARP-Rollout stopping schedules remain an open evaluation.

Further training-dynamics, supervision-duration, and structural-novelty analyses are provided in the revised appendix.

AuthorsOpenReview ↗

Response to Reviewer GzJP – Q3: Enzyme Geometry Definitions

We thank the reviewer for identifying this inconsistency. The wording in Appendix E.3 was misleading and has been corrected. The backbone reward, the BB RMSD column in Table 2, and the backbone success criterion all use the same RF3-refolded backbone self-consistency RMSD.

Specifically, the metric is defined as follows:

  • Structures: The generated protein structure and its RF3-refolded structure, rather than the prescribed catalytic-motif target geometry.
  • Atoms: The full protein backbone (N, Cα, C, O), not just the catalytic motif.
  • Alignment: Optimal rigid-body superposition using the same backbone atoms over which RMSD is computed.
  • Evaluation: Table 2 reports the median RMSD over 100 designs, with a backbone success threshold of ≤ 2.0 Å.

Thus, these are not distinct metrics. We have revised it in Appendix E.3.

Reviewer_pLVwOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Search-Augmented Reinforcement Post-Training (SARP) for adapting pretrained all-atom protein generative models to downstream design objectives. SARP combines endpoint-based DGPO with auxiliary flow-matching supervision on high-reward samples obtained through reward-guided beam search. The auxiliary supervision can be disabled after an initial training phase. The method is evaluated on small-molecule binder design and target-specific enzyme design, showing improvements in computational quality metrics and the number of structurally distinct designs passing the evaluation criteria. Additional experiments examine supervision sources, search configurations, training dynamics, and cross-evaluator consistency using AlphaFold3.

Critical strengths

(1) Relevant and practically motivated problem. Improving the yield of high-quality designs from pretrained protein generators is an important objective. Distilling search-discovered samples into the generator provides a reasonable approach to improving subsequent generation, while endpoint-based optimization avoids retaining complete denoising trajectories. (2) Improvements over the pretrained model and the RL-only baseline. Across the five enzyme targets, SARP increases the average computational pass rate from 15.0% for the pretrained model and 54.4% for DGPO to 69.4%. The average number of distinct passing structural clusters per 100 designs increases from 11.2 and 35.6 to 42.2, respectively. Evaluating both success rate and structural diversity strengthens the empirical assessment. (3) Useful ablations of the proposed framework. The paper compares SARP with DGPO, pure search-based supervised fine-tuning, larger rollout groups, and supervision using high-reward ordinary rollouts. These experiments help distinguish the contributions of reinforcement, endpoint supervision, and the source of supervised samples.

Critical weaknesses

(1) The evaluation scope is limited and does not sufficiently establish generalization. In the enzyme-design experiments, a separate model is fine-tuned for each target and evaluated on the same target. This setup establishes target-specific adaptation, but does not demonstrate that the learned design strategies transfer to unseen enzyme targets. In the small-molecule binder experiments, post-training spans many ligands, yet evaluation covers only four ligand identities, with FAD and SAM also appearing in the post-training dataset. The authors acknowledge that this benchmark follows prior work and is not a strict unseen-ligand evaluation. Nevertheless, its limited size and ligand overlap constrain the evidence for cross-ligand generalization. A larger evaluation with chemically disjoint held-out ligands, together with cross-target transfer experiments for enzyme design, would substantially strengthen the paper. (2) The efficiency comparison does not account for the cost of retraining for each enzyme target. Since the enzyme experiments require separate fine-tuning for every new design target, post-training is part of the computational cost of solving that target-specific design problem. Standard sampling after fine-tuning may be cheaper than inference-time search, but this alone does not establish an overall computational advantage.

Questions for authors

(1) Can the authors evaluate generalization to a broader set of unseen small-molecule targets? The current four-ligand benchmark, which includes ligand identities present during post-training, provides limited evidence of cross-ligand generalization. (2) Can a model post-trained multiple enzyme targets improve design performance on held-out enzyme targets without additional fine-tuning? Since the current experiments optimize each enzyme target independently, the transferability of the learned design strategies remains unclear. (3) If effective adaptation requires separate post-training for each new target, comparisons of computational efficiency with inference-time scaling methods should account for SARP’s target-specific fine-tuning cost. At what generation volume does the lower inference cost of the post-trained model offset its additional training cost?

6 comments
AuthorsOpenReview ↗

Response to Reviewer pLVw - Weakness 1 and Q1: Generalization to Held-Out Ligands

We thank the reviewer for raising this question. Our original four-ligand benchmark follows the RFDiffusion-AA setup, including two held-out and two in-distribution ligands. To further assess cross-ligand generalization, we evaluate SARP on five additional held-out ligands (MTX, OHT, CEF, CZH, and HEB), none of which appears in the post-training dataset, following the same evaluation protocol as the original benchmark.

Generalization to Held-Out Ligands

LigandMethodΔG ↓min-iPAE ↓scRMSD ↓ligRMSD ↓pLDDT ↑meanTM ↓
MTXBase model2.247.974.609.2365.430.329
GRPO−4.228.534.3610.6367.240.340
DGPO−0.276.741.689.5375.850.405
Search-SFT2.865.851.2610.8977.220.553
SARP−7.626.361.209.8378.740.417
OHTBase model−2.587.904.3710.5565.380.335
GRPO−8.857.514.1410.7867.460.332
DGPO−8.056.052.4512.2272.730.349
Search-SFT−11.844.341.217.9578.400.592
SARP−18.933.281.355.7378.970.414
CEFBase model0.725.594.167.5471.850.331
GRPO−2.455.802.767.8872.460.338
DGPO−0.254.301.659.3379.390.471
Search-SFT1.063.421.279.9381.130.583
SARP−5.843.071.056.3582.210.416
CZHBase model−8.757.045.069.4666.010.332
GRPO−12.558.054.909.5364.840.329
DGPO−9.735.502.179.1574.820.373
Search-SFT−13.123.461.227.3079.460.563
SARP−23.922.951.236.7680.750.429
HEBBase model−8.845.024.389.5265.360.334
GRPO−13.464.874.1310.4963.900.350
DGPO−9.424.213.0910.2171.760.355
Search-SFT−9.634.071.4111.2977.310.566
SARP−13.662.921.478.5878.230.397
AverageBase model−3.446.704.519.2666.810.332
GRPO−8.316.954.069.8667.180.338
DGPO−5.545.362.2110.0974.910.391
Search-SFT−6.134.231.279.4778.700.571
SARP−13.993.721.267.4579.780.415

ΔG is measured in REU; min-iPAE, scRMSD, and ligand RMSD are measured in Å. Lower meanTM indicates greater structural diversity.

Results. SARP achieves the best average performance across all five binding- and structure-quality metrics, improving binding energy from −3.44 to −13.99 REU, min-iPAE from 6.70 to 3.72 Å, and scRMSD from 4.51 to 1.26 Å relative to the pretrained model. Notably, SARP achieves the best binding energy on all five held-out ligands and the best min-iPAE on four of five.

These results provide additional evidence that SARP's post-training improvements transfer to unseen ligand identities without target-specific fine-tuning.

AuthorsOpenReview ↗

Response to Reviewer pLVw - Weakness 1 and Q2: Generalization on Multi-Target Enzyme Post-Training

We thank the reviewer for this question so that we can further solidify our experiments. Our initial target-specific enzyme post-training experiments were designed to assess SARP's ability to address challenging enzyme-design problems under target-specific catalytic constraints. To further investigate generalization, we post-train a single model on diverse ligand-conditioned motif-scaffolding tasks till converge and evaluate it on five held-out enzyme targets without additional fine-tuning, referred to as Pooled SARP. Further details on the pooled post-training setup, dataset construction, and complete evaluation results are provided in Appendix J.

TargetMethodStepsCat. match ↑Cat. RMSD ↓BB RMSD ↓Lig. clash ↓Pass ↑Unique pass ↑
M0157Base—0.9992.741.770.001%18%17
Target-specific DGPO1701.0001.660.630.001%57%33
Target-specific SARP90(30)1.0001.340.580.003%74%43
Pooled SARP830.9971.830.660.007%31%27
M0552Base—0.9983.091.520.004%11%7
Target-specific DGPO1200.9981.620.740.015%60%40
Target-specific SARP90(30)1.0001.360.570.003%72%54
Pooled SARP831.0001.840.690.007%27%21
M0732Base—0.9982.951.490.008%20%17
Target-specific DGPO1000.9901.530.970.021%62%42
Target-specific SARP60(20)1.0001.110.580.006%89%44
Pooled SARP830.9962.221.020.032%21%18
M0904Base—1.0002.391.280.000%21%12
Target-specific DGPO1200.9991.610.990.005%75%47
Target-specific SARP140(20)1.0001.200.600.000%88%52
Pooled SARP830.9991.650.970.003%61%35
M0375Base—0.9993.181.490.011%5%3
Target-specific DGPO1101.0002.261.150.006%18%16
Target-specific SARP80(20)1.0001.910.630.008%24%18
Pooled SARP831.0002.490.910.010%6%6

Cat. RMSD and BB RMSD are measured in Å. Unique pass denotes the number of distinct passing structural clusters per 100 designs.

Interpretation. Without target-specific fine-tuning, Pooled SARP improves average pass rate from 15.0% to 29.2% and unique passing clusters from 11.2 to 21.4 across five held-out enzyme targets, while reducing catalytic-site RMSD (2.87 → 2.01 Å) and backbone RMSD (1.51 → 0.85 Å). Improvements vary by target, with M0904 showing the largest pass-rate gain (21% → 61%).

Target-specific SARP achieves stronger performance (69.4% pass rate, 42.2 unique clusters), which is expected given the heterogeneous catalytic geometries and stringent, target-dependent constraints of enzyme design.

Overall, these results provide evidence that pooled post-training can transfer improvements to unseen enzyme-design targets without additional fine-tuning, while target-specific optimization remains beneficial. Full experimental details are provided in Appendix J.

AuthorsOpenReview ↗

Extra Response: Extended Experiments on Generalization to DNA-Binding Protein Design

To further evaluate the generality of SARP beyond small-molecule binding and enzyme design, we extend our framework to DNA-binding protein design, involving a distinct class of protein–nucleic acid interactions.

Following the newly published powerful partially-latent protein generation model RFD4-Proteina (https://rfd4-proteina.org/) setup, we post-train the model on 603 protein–DNA complexes containing DNA targets of 10–14 base pairs and evaluate on three held-out DNA-binding targets (7m5w, 7n5u, and 7rte). We compare SARP against the pretrained RFD4-Proteina and DGPO.

Generalization results on held-out DNA-binding targets.

TargetMethodmin-iPAE ↓ (Å)BB RMSD ≤ 8 Å ↑ (%)ipTM ↑ (×100)Unique pass ↑ (%)
7m5wBase5.006.055.642.0
DGPO3.659.067.143.0
SARP2.8028.071.2510.0
7n5uBase5.3818.059.973.0
DGPO3.5329.067.267.0
SARP2.8636.070.407.0
7rteBase5.9124.058.875.0
DGPO3.0731.070.728.0
SARP3.6837.067.148.0
AverageBase5.4316.058.163.33
DGPO3.4223.068.376.00
SARP3.1233.769.598.33

Results. Across the three held-out targets, SARP achieves the best average performance on all four metrics, reducing min-iPAE from 5.43 to 3.12 Å and increasing the DNA-aligned backbone RMSD pass rate from 16.0% to 33.7% compared with the pretrained model. SARP also improves the unique pass rate from 3.33% to 8.33%, outperforming DGPO (6.00%).

These results provide additional evidence that SARP generalizes beyond protein–ligand interactions to protein–DNA binding design, achieving improved binding confidence, structural consistency, and diversity of successful designs on held-out targets.

Reviewer_uxAyOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SARP improves pretrained all-atom protein generators by combining trajectory-free reinforcement learning with supervised learning from high-reward designs discovered through beam search. This allows the model to learn from expensive search and generate better proteins without search at inference time. Experiments on small-molecule binder and enzyme design show improvements in design quality and structural diversity over existing baselines. The authors also provide cross-evaluator validation and gradient analyses to examine the effects of search supervision.

Critical strengths

Convincing motivation: I think improving standard-sampling protein generation by distilling expensive search into the model is a convincing motivation. Empirical improvements: SARP demonstrates meaningful improvements in protein-design quality and structural diversity over existing baselines. Useful ablations: I appreciate the comparisons between search-derived and rollout-derived supervision, particularly their effects on structural diversity. Additional analytical results

Critical weaknesses

Limited methodological novelty in light of missing references in this work (look at the Missing References section below): SARP mainly combines existing DGPO, beam search, and reward-selected supervision. I am not entirely convinced that the combination offers sufficient novelty over existing reward-guided fine-tuning and distillation methods. Insufficient isolation of search benefits: Although the authors compare SARP with SARP-Rollout, the additional exploration provided by search is not controlled for. It remains unclear whether structured search is necessary or whether additional high-reward samples would produce similar improvements. Unclear computational efficiency when the authors claim compute efficiency: The paper demonstrates improvements without inference-time search but does not adequately quantify the additional training cost of beam search and reward evaluations. Limited generalization to OOD/unseen samples: The binder evaluation covers only four ligands, while enzyme models are optimized separately for individual targets. I find the evidence for generalization to unseen design conditions limited. However, this is a minor issue that did not affect my final assessment of this paper Missing References

Su et al. (2026), Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Design. Closely related reward-guided distillation for biomolecular generation. https://arxiv.org/abs/2507.00445 Zheng et al. (2026), DiffusionNFT: Online Diffusion Reinforcement with Forward Process. Relevant trajectory-free diffusion post-training approach. https://arxiv.org/abs/2509.16117 Anil et al. (2026), Fine-Tuning Diffusion Models via Intermediate Distribution Shaping. Relevant rejection-sampling-based diffusion fine-tuning method. https://proceedings.iclr.cc/paper_files/paper/2026/hash/ee0ca8e48b694b7f183ad855da0b3868-Abstract-Conference.html

Questions for authors

Q1) How does SARP fundamentally differ from existing reward-guided diffusion fine-tuning and distillation methods, particularly those mentioned in the Missing References I have provided above? Q2) Could the authors compare SARP against a compute-matched best-of-N supervision baseline? I think this would help isolate whether structured beam search provides benefits beyond additional sampling. Q3) Could the authors report training GPU-hours, sampling costs, and reward-evaluation calls? Since efficiency is a central motivation, I think an amortized training-versus-inference cost comparison would be helpful. Q4) Could the authors analyze whether search-derived endpoints provide greater structural novelty or simply higher rewards compared with ordinary rollout endpoints? Q5) Could the authors provide evidence of generalization to unseen ligands or catalytic motifs without additional target-specific post-training? I believe this will be important for generalization purposes.

5 comments
AuthorsOpenReview ↗

Response to Reviewer uxAy - Q1: Methodological Novelty and distinction from other works

We thank the reviewer for highlighting these closely related works. We agree that DGPO, beam search, and reward-guided distillation are individually established techniques. Our contribution is not a new diffusion RL objective, but an effective post-training framework for pretrained all-atom protein structure generators, whereas most prior protein post-training studies have focused on sequence-level generation.

Specifically, we aim to address two challenges: (1) adapting RL to all-atom protein generation, including constructing reliable structure-based functional rewards, and (2) effectively distilling the benefits of reward-guided search into the generator itself, so that improvements can be realized through standard sampling without additional inference-time search.

Distinction from prior work.

We have carefully examined the works highlighted by the reviewer and clarify the following distinctions:

  • VIDD (Su et al.).

    1. VIDD primarily investigates sequence-level post-training and uses intermediate denoised predictions to approximate reward-improved policies through value-guided iterative distillation. In contrast, SARP targets joint all-atom protein structure generation, where functional rewards depend on detailed three-dimensional geometry.
    2. Directly evaluating intermediate clean-structure predictions in our setting can be unreliable because they may contain backbone discontinuities, steric clashes, or incorrect protein–ligand geometry, making metrics such as Rosetta binding energy noisy. SARP addresses this by completing denoising trajectories before reward evaluation, using intermediate states as branching points for search rather than scoring potentially invalid intermediate predictions. This enables reward selection based on fully generated, structurally evaluated candidates.
  • DiffusionNFT (Zheng et al.). DiffusionNFT proposes an alternative diffusion RL optimization objective that learns from positive- and negative-reward samples through forward-process flow matching. It is therefore more directly comparable to the DGPO optimization component of SARP than to our overall framework. While DiffusionNFT focuses on how to update a diffusion policy using reward signals, SARP additionally investigates how to discover better training examples through reward-guided intermediate-state search. These components are conceptually complementary, and our search-augmentation strategy is not inherently tied to DGPO.

  • P-GRAFT (Anil et al.). P-GRAFT selects high-reward trajectories and uses their intermediate noisy states for supervised fine-tuning, sharing a similar motivation with SARP's Search-SFT component. However:

    1. P-GRAFT selects from existing trajectories, but SARP actively explores additional high-reward structures through intermediate-state branching search and distills them alongside on-policy RL.
    2. Our SARP-Rollout ablation further shows that selecting high-reward rollout samples alone yields substantially fewer unique passing structures than SARP (21.0 vs. 42.2 clusters per 100 designs), highlighting the benefit of search-derived supervision in our setting.

Overall, our contribution is a problem-driven framework integrating on-policy RL, full-rollout structural evaluation, intermediate-state search, and search-derived supervision to improve the standard-sampling performance of all-atom protein generators instead of just protein sequences. We have revised the manuscript to explicitly discuss these related works and clarify the distinctions above.

AuthorsOpenReview ↗

Response to Reviewer uxAy - Q2: Oracle-Budget-Matched Best-of-N Baseline

We thank the reviewer for suggesting a compute-matched Best-of-N baseline. Prior work on Proteina-Complexa (https://arxiv.org/abs/2603.27950) has investigated both independent Best-of-N sampling and structured beam search, showing that their relative effectiveness depends on task difficulty. They show that Best-of-N can be highly effective on easier targets and structured search becomes more advantageous on challenging protein design tasks. To isolate the contribution of structured search from additional sampling, we introduce SARP-Best-of-N, which replaces reward-guided beam search with independent Best-of-N sampling.

Compute matching.

We match the number of reward-oracle calls per rank per optimization step. At beam width W=8W=8, SARP uses 32 standard DGPO rollouts and 136 additional search evaluations. SARP-Best-of-N therefore evaluates 136 independently generated candidates alongside the same 32 DGPO rollouts, matching the total budget of 168 oracle calls. Similarly, at W=2W=2, both methods use 66 oracle calls, with 34 additional independent candidates for Best-of-N. In our measurements, Best-of-N also requires more GPU computation than SARP, making this comparison conservative in terms of computational cost.

Enzyme design.

We evaluate both methods on five enzyme targets under matched reward-oracle budgets and training schedules. SARP produces more unique passing structural clusters across all five targets, achieving 211 versus 144 target-wise unique passing clusters in aggregate (46.5% improvement). While Best-of-N achieves competitive or better geometric quality and pass rates on several targets, SARP consistently generates a more structurally diverse set of successful designs.

TargetMethodStepsCat. match ↑Cat. RMSD ↓BB RMSD ↓Lig. clash ↓Pass ↑Unique pass ↑
M0732Base—0.9982.951.490.008%20%17
Best-of-N60(20)1.0001.030.520.006%92%22
SARP60(20)1.0001.110.580.006%89%44
M0375Base—0.9993.181.490.011%5%3
Best-of-N80(20)1.0001.900.660.000%20%4
SARP80(20)1.0001.910.630.008%24%18
M0552Base—0.9983.091.520.004%11%7
Best-of-N90(30)1.0001.430.600.0029%63%48
SARP90(30)1.0001.360.570.003%72%54
M0157Base—0.9992.741.770.001%18%17
Best-of-N90(30)1.0001.320.560.0040%77%42
SARP90(30)1.0001.340.580.003%74%43
M0904Base—1.0002.391.280.000%21%12
Best-of-N140(20)1.0000.970.550.0014%96%28
SARP140(20)1.0001.200.600.000%88%52

These results demonstrate that stronger geometric recovery or higher pass rates do not necessarily translate into greater diversity among successful designs. SARP's advantage in enzyme design extends beyond additional independent sampling under an equal reward-oracle budget.

Small-molecule binder design.

We additionally compare SARP with an oracle-budget-matched Best-of-N baseline at training step 168, which is the farthest we can get due to the time limitation. SARP (W=2W=2) outperforms Best-of-N (N=34N=34) across all six reported metrics, including binding energy, interface confidence, structural quality, and diversity.

MethodStepΔG ↓min-iPAE ↓pLDDT ↑scRMSD ↓Lig. RMSD ↓Mean TM ↓
SARP-Best-of-N (N=34)168−10.793.9578.101.317.520.414
SARP (W=2)168−11.063.8778.581.256.790.410

The improvements are relatively modest in small-molecule binder design but more pronounced in enzyme design, consistent with prior observations that structured search becomes especially useful on challenging targets.

Overall, these experiments provide evidence that structured search offers benefits beyond increasing the number of independently sampled candidates, particularly by improving the structural diversity of successful enzyme designs. We further examine the structural novelty of search-derived endpoints in our response to Q4.

AuthorsOpenReview ↗

Response to Reviewer uxAy – Q3: Computational Cost

We thank the reviewer for this question. We quantify SARP's computational cost through generator function evaluations (NFEs), reward-oracle calls, measured GPU-hours, and training–inference amortization. We have also updated the manuscript accordingly.

1. Generator evaluations and reward-oracle calls.

With 32 rollouts and 400 denoising steps, DGPO requires 12,800 generator NFEs and 32 reward-oracle calls per rank-step. SARP additionally performs four rounds of beam search, including branch advancement and completion for reward evaluation, requiring 12,800+4,000W12,800+4,000W total NFEs per rank-step. At W=2W=2 and W=8W=8, this corresponds to 1.625× and 3.50× generator NFEs, respectively, and 66 (2.06×) and 168 (5.25×) reward-oracle calls.

2. Measured training GPU-hours.

We report GPU-hours per optimization step using eight GPUs per run:

DomainMethodGPU-h / step
EnzymeDGPO3.12
EnzymeSARP-Rollout2.60
EnzymeSARP (W=2)4.10
EnzymeSARP (W=8)10.04
sm-binderDGPO3.34
sm-binderSARP-Rollout2.87
sm-binderSARP (W=2)3.65
sm-binderSARP (W=8)7.44

These measurements quantify the training overhead of search supervision. Complete training costs, including subsequent DGPO-only optimization, are reported in Appendix K.

3. Amortized training-versus-inference cost.

Inference-time beam search requires 10.84× more GPU-hours per generated design than standard sampling on L40S GPUs (0.06374 vs. 0.00588 GPU-h/design). We account for the full target-specific post-training cost when calculating the break-even point:

TargetTrain GPU-hSARP GPU-h/passBase+search GPU-h/passBreak-even passes
M03752400.02450.3984642
M07322750.006610.08293,605
M05525130.008170.07997,154
M09044850.006680.07457,156
M01575280.007950.07058,440

Target-specific training–inference amortization. GPU-h/pass denotes marginal inference cost per passing design.

Across five enzyme targets, SARP requires 8.9–16.3× fewer inference GPU-hours per passing design, with its upfront training cost amortized after 642–8,440 passing designs (approximately 4,100–9,100 generated candidates when comparing equal generation volumes).

These scales are relevant to practical protein design campaigns. Previous studies have generated approximately 10,000–50,000 computational candidates per target, including Ras isoform-selective binder design and intrinsically disordered protein binder design. Thus, SARP's upfront post-training cost can be amortized within the scale of large target-specific computational design campaigns.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Reviewer Concerns and Our Responses

We thank all reviewers for their thoughtful feedback and constructive suggestions. We are encouraged that the reviewers recognize:

  • The importance and practical motivation of adapting pretrained all-atom protein generators to real-world protein design requirements through post-training and distilling expensive search into efficient standard sampling (Reviewers uxAy, pLVw).
  • Meaningful improvements in design quality and structural diversity (Reviewers uxAy, pLVw, GzJP). In particular, Reviewer GzJP highlights that our improvements extend beyond average reward to generating more structurally distinct successful designs.
  • The value of our ablation studies and additional analyses, particularly the comparison between search-derived and rollout-derived supervision (Appendix S), together with cross-evaluator validation (Appendix L) (Reviewers uxAy, pLVw, GzJP).

We have addressed the remaining questions through additional experiments, computational analyses, and manuscript revisions, summarized below:

  • Computational cost and efficiency (uxAy Q3; pLVw Q3; GzJP Weakness 1, Q1): We provide detailed accounting of generator evaluations, reward-oracle calls, and training GPU-hours (Appendix K, Tables 9–12). We also compare SARP against inference-time search, demonstrating 10.84× lower inference GPU-hours per generated design and 8.9–16.3× lower marginal inference cost per passing design across five enzyme targets (Appendix K.4, Tables 13–14).

  • Generalization to additional design tasks (uxAy Q5; pLVw Weakness 1, Q1–Q2): We expand our evaluation beyond the original benchmarks:

    • Small-molecule binders: Evaluation on five additional held-out ligands (Appendix I, Table 7).
    • Enzyme design: Pooled post-training on diverse ligand-conditioned motif-scaffolding tasks, evaluated on five held-out enzyme targets without further fine-tuning. Pooled SARP improves average pass rate from 15.0% to 29.2% and unique passing clusters from 11.2 to 21.4 per 100 designs (Appendix J, Table 8).
    • DNA-binding proteins: Evaluation on three held-out DNA-binding targets, extending SARP to protein–DNA interface design and improving successful, structurally distinct designs (Section 4.3, Table 3).
  • Additional analyses of search-specific benefits (uxAy Weaknesses, Q1–Q2, Q4; GzJP Weakness 1, Q1): We clarify SARP's methodological contribution and distinguish it from related reward-guided fine-tuning methods (Appendix A). We provide an oracle-budget-matched Best-of-N baseline in our reviewer responses and analyze the structural novelty of search-derived endpoints (Appendix S). These results support the benefits of structured search beyond increased sampling or reward-selected ordinary rollouts, particularly for the diversity of successful enzyme designs.

  • Additional methodological clarifications and revisions (GzJP Weakness 2, Q2–Q3): We examine training variability through an additional SARP seed on M0552, obtaining comparable pass rates (72% vs. 76%) and unique passing clusters (54 vs. 51). We clarify supervision duration and checkpoint selection (Appendix G.9), provide supporting gradient and supervision-duration analyses (Appendices Q and R), further analyze SARP-Rollout training dynamics (Appendix S), and correct the enzyme backbone RMSD description (Appendix E.3).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.