acceptodds
Under review as a conference paper at ICLR 2027

ProEdit: One-Step Protein Optimization with Diffusion Protein Language Models

Abstract

Protein design often begins downstream optimization from a complete candidate sequence, yet reward-guided optimization of diffusion-pretrained protein models typically retains the native iterative denoising process. We ask whether this generative trajectory is necessary once a candidate already exists. We introduce ProEdit, which repurposes a diffusion-pretrained protein language model as a target-specific protein sequence optimizer. Rather than optimizing through the native denoising process, the editor conditions on the complete parent sequence and samples a complete edited sequence as a single parallel action. This policy admits explicit action likelihoods and an exact conditional KL to a frozen reference policy, enabling direct reward learning without reconstructing diffusion trajectories. We train the editor from black-box structural rewards using parent-wise group-relative reinforcement learning, where candidates sampled from the same parent provide the relative optimization signal. On protein-binder optimization against PD-L1 and IFNA2, ProEdit substantially outperforms native diffusion-rollout GRPO in reward and iPTM while reducing training GPU-hours by by a factor of approximately 27 under matched reward-evaluation and optimizer-update budgets, and also outperforms reward-weighted supervised training. Most of the improvement is already present in raw policy proposals before reward-based selection, and the gains persist under independent structure predictors. These results show that diffusion pretraining can provide a strong initialization for downstream protein optimization without prescribing the decision process used to optimize complete candidate sequences.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein language models, Diffusion protein language models, Reinforcement learning, Protein sequence editing, Protein design

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bG3kOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ProEdit, a one-step protein sequence optimization method based on a diffusion-pretrained protein language model. Instead of using the original multi-step diffusion process, ProEdit takes a complete parent sequence as input and directly generates an edited sequence in one step. The model is trained with GRPO using black-box structural rewards, with a reference KL to control policy updates. They evaluate ProEdit on two protein binder optimization tasks, PD-L1 and IFNA2. The results show improvements in structural rewards and iPTM compared with pretrained editing, diffusion-based GRPO, and reward-weighted supervised training. ProEdit also reduces training cost significantly. Additional experiments with independent structure predictors, sequence diversity, and parent conditioning further support the effectiveness of the method.

Critical strengths
  1. Strong experimental results and high efficiency. ProEdit achieves clear improvements in structural rewards and iPTM on two protein binder optimization tasks. It also greatly reduces training cost compared with the native diffusion rollout baseline. These results show the potential practical value of the proposed approach.

  2. Comprehensive experimental analysis. The authors provide several additional experiments, including raw proposal quality, parent conditioning, sequence diversity, and evaluation with independent structure predictors. These analyses provide evidence for the effectiveness of the learned editing policy.

Critical weaknesses
  1. Limited methodological novelty compared with existing work. Several key ideas of this paper have already been explored in related studies. RIDER [1] uses reinforcement learning with external structural rewards to optimize diffusion-based RNA sequence generation. This already covers the idea of using black-box structural feedback to directly optimize diffusion-based sequence models. SOLD [2] goes further by proposing single-step RL optimization, where a complete RNA sequence is predicted and evaluated using external structural rewards without sampling the full diffusion trajectory during training. It also uses policy optimization and KL regularization, which are similar to the main optimization ideas in ProEdit. I understand that RIDER and SOLD focus on RNA inverse folding, while ProEdit studies protein optimization. However, they all belong to biological sequence design and optimization, and share similar ideas of using diffusion models and external structural rewards. ProEdit differs from these works by taking a complete parent sequence as input and directly generating an edited sequence without diffusion trajectories during training or inference. However, this remaining difference mainly comes from a simple parallel categorical policy combined with existing GRPO optimization. I find the methodological novelty relatively limited, as the paper does not clearly explain what new insights or advantages this design provides beyond existing approaches.

  2. Unclear necessity of diffusion in the proposed framework. The paper is titled "One-Step Protein Optimization with Diffusion Protein Language Models", but I do not see why diffusion is necessary for the proposed method. ProEdit does not optimize or use the diffusion trajectory during downstream training or inference. Instead, it mainly uses the diffusion-pretrained model as a sequence prediction network to directly generate edited sequences from a given parent. In this case, why is a diffusion model needed? For example, could autoregressive or masked protein language models also be used for sequence optimization with similar reward-based training? The proposed parallel prediction and GRPO optimization do not seem to rely on any diffusion-specific mechanism. The paper does not provide sufficient explanation or experimental evidence showing what unique advantage diffusion pretraining brings to this framework. This makes the motivation and methodological contribution less appealing.

References

[1] Hu et al., "RIDER: 3D RNA Inverse Design with Reinforcement Learning-Guided Diffusion." ICLR 2026.

[2] Si et al., "Structure-based RNA Design by Step-wise Optimization of Latent Diffusion Model." AAAI 2026.

Questions for authors
  1. What is the main novelty compared with existing studies? Please see weakness1. Could the authors clarify what the main new contribution of ProEdit is beyond these existing ideas?

  2. Why is diffusion pretraining necessary for ProEdit? The proposed method does not use diffusion trajectories during downstream optimization or inference. Could other pretrained protein language models, such as autoregressive or masked models, achieve similar results with the same editing policy and GRPO training? What specific advantage does diffusion pretraining provide?

  3. Does the current comparison really demonstrate the advantage of one-step editing? According to Table 8 in Appendix D.1, ProEdit takes a complete parent sequence as input, while FullRollout-GRPO starts from an all-mask sequence with only the sequence length matched. This means the comparison changes both the input information and the generation process at the same time. The large performance improvement may therefore come from parent conditioning rather than the one-step editing policy itself. Could the authors explain how these two factors are separated in the current experiments? Would ProEdit still show similar advantages when compared with a parent-conditioned multi-step diffusion editor under the same training budget?

6 comments

Thank you for your thoughtful review and for recognizing the strong experimental results, computational efficiency, and comprehensive evaluation of ProEdit. We appreciate your questions regarding methodological novelty, the role of diffusion pretraining, and the comparison with parent-conditioned iterative methods. Below, we clarify our contributions in relation to SOLD and RIDER and present additional experimental evidence addressing these concerns.

1. Differences from SOLD and RIDER and our main contributions (W1, Q1)

We would first like to clarify the methodological distinctions between ProEdit, SOLD, and RIDER. Simply grouping these methods as applications of RL to diffusion models does not capture the distinct problems they address or their methodological contributions.

Diffusion models and probabilistic formulations

RIDER and SOLD share a continuous Gaussian diffusion formulation, although they operate in different representation spaces. ProEdit differs fundamentally in its underlying generative formulation. DPLM-2 is a pretrained discrete masked-diffusion protein language model. Its pretraining operates on discrete protein tokens, uses masking for corruption, and learns to predict masked tokens, rather than predicting Gaussian noise in continuous representations. For downstream editing, ProEdit directly adapts this pretrained model to define a categorical policy over complete amino-acid sequences conditioned on the parent.

In particular, SOLD uses pretrained RNA-FM representations to construct a separate latent diffusion generator, whereas ProEdit directly reuses the pretrained diffusion protein language model itself for reward-driven protein optimization.

RL policies and actions

RIDER and SOLD both apply RL to the native denoising policies of pretrained diffusion models for structure-conditioned RNA inverse folding. RIDER samples complete denoising trajectories and uses terminal structural rewards to update the policy. SOLD avoids full-trajectory sampling during training by sampling noise levels and evaluating denoising predictions: a short-term reward for a single transition (t→t−1t\rightarrow t-1), and a long-term reward for a direct prediction (t→0t\rightarrow 0). Both provide reward feedback to the same noise-conditioned denoising policy, and inference retains the native multistep denoising process.

ProEdit takes a different approach. Rather than optimizing DPLM-2's native denoising policy, we repurpose the pretrained model to define a parent-conditioned policy over complete edited sequences. The complete edited sequence is the RL action, evaluated using its target-specific structural reward. This same action is optimized during training and executed in a single forward pass at inference. It is neither a single transition nor a shortened version of the native diffusion process.

Downstream optimization tasks

RIDER and SOLD address RNA inverse folding, whereas ProEdit optimizes existing protein candidates. Given a parent binder and a target-specific structural reward, ProEdit learns to directly improve the candidate's binding quality.

This task difference motivates a different policy design. RIDER and SOLD condition sequence generation on the target structure, whereas ProEdit conditions editing on the complete parent sequence and evaluates the resulting candidate using its target-specific reward. We therefore decouple input visibility from output editability: the complete parent remains available as context, while every output position can be modified. This defines a direct sequence-editing policy for downstream protein optimization, rather than retaining the native multistep denoising process.

Main contributions

  1. A direct, full-context policy for downstream protein optimization. We turn a diffusion-pretrained protein language model originally used for multistep generation into a one-step sequence optimizer conditioned on the complete parent sequence, decoupling input visibility from output editability.

  2. Direct reward learning with tractable policy quantities. We use parent-wise GRPO to learn complete sequence edits directly. The policy provides explicit action likelihoods and an exact conditional sequence KL to a frozen reference policy. The accompanying theoretical analysis connects reward training to the action used at inference.

  3. Effective and compute-efficient learned protein optimization. Across two binder-design benchmarks, ProEdit achieves the highest reward and iPTM among the methods compared. Its raw policy proposals already show high reward. ProEdit also substantially reduces training cost relative to rollouts with the native 500-step diffusion decoder. The additional parent-conditioned iterative baselines further support the effectiveness of direct reward training for editing.

Diffusion pretraining provides a foundation for protein modeling without prescribing the decision process used for downstream reward optimization.

2. Why use diffusion pretraining, and can other PLMs be used? (W2, Q2)

We chose DPLM-2 because masked diffusion pretraining equips it with strong capabilities in protein sequence modeling, structure-aware representation learning, and conditional generation. DPLM-2 can leverages bidirectional context to predict residue distributions in parallel, providing a strong pretrained foundation for a full-context editing policy. We investigate whether these pretrained capabilities can be transformed through downstream reward training into the ability to directly optimize existing protein candidates without retaining the native multistep denoising process.

To this end, we retain DPLM-2's pretrained parameters and residue prediction capability, use the complete parent sequence as the policy input, and define the complete edited sequence as the action to be optimized. Pretraining provides a prior for protein modeling, while RL learns effective modifications for the target. This changes not only the sampling strategy, but also the way the pretrained model is trained and optimized for downstream tasks.

Experiments validate this design. Frozen DPLM-2 achieves one-step editing rewards of 0.2597 and 0.2532 (PD-L1 and IFNA2, respectively), which increase to 0.9787 and 0.9434 after GRPO training. This shows that high-quality one-step optimization is not an inherent capability of the pretrained model, but can be learned directly through reward feedback on complete edited sequences. The comparison with FullRollout-GRPO further indicates that retaining the native multistep denoising process is not necessary for this downstream optimization task.

As suggested, we also applied the same full-context editing interface and GRPO training to ESM-2 and ESM-3. Each model used 6,400 optimizer updates and 409,600 training candidates. We followed the sampling and evaluation protocols in Table 1, except that we disabled parent fallback.

Pretrained backbonePD-L1 RewardIFNA2 Reward
DPLM-2(650M)0.97870.9434
ESM-2(650M)0.839570.57184
ESM-3(1.4B)0.854030.67404

These results suggest that the editing formulation is applicable across different pretrained PLMs. Under the evaluated training setup and budget, DPLM-2 achieves higher Reward, supporting its use as the initialization in this study. More broadly, DPLM-2 provides a strong pretrained foundation for downstream protein optimization without requiring its native iterative denoising process to be retained as the downstream policy.

3. Parent-conditioned iterative baseline (Q3)

We trained three parent-conditioned GRPO baselines. M64 starts from a partially masked parent and iteratively completes the masked positions while keeping observed residues fixed. P64 and P500 use the complete parent as a fixed prefix and recover a fully masked part through 64 and 500 native denoising steps, respectively, which providing direct parent-conditioned rollout comparisons with ProEdit.

All methods used the same training budget of 3.2K optimizer updates and 204.8K training reward evaluations. Sampling and evaluation followed the protocol in Table 1, except that parent fallback is disabled.

MethodPD-L1 RewardIFNA2 RewardPD-L1 Neff,90N_{\mathrm{eff},90}IFNA2 Neff,90N_{\mathrm{eff},90}Training GPU-hours (PD-L1 / IFNA2)
ProEdit0.93320.8719200.00200.0021.6 / 33.1
M640.25290.2376200.00200.00110.0 / 120.6
P640.25900.35031.043.58165.3 / 175.1
P5000.24380.32442.915.061,303 / 1,321

All three parent-conditioned iterative baselines still trail ProEdit in reward. M64 retains high sequence-family diversity but achieves low reward, whereas P64 and P500 exhibit substantial family concentration. Increasing the number of denoising steps from 64 to 500 does not improve reward, despite substantially higher training cost. Under the same training budget, ProEdit achieves higher reward while maintaining (Neff,90=200N_{\mathrm{eff},90}=200) on both targets, with considerably fewer GPU-hours.

We also trained a GRPO baseline with masked infilling (Masked GRPO). During training, we randomly masked positions in the parent sequence, kept visible residues fixed, and predicted the masked positions in a single forward pass. This followed the masked-corruption and recovery formulation of diffusion pretraining, with predictions restricted to the masked positions.

The table reports inference results using the native 500-step diffusion decoder starting from an all-mask input.

TargetInference samplingOptimizer updatesReward
PD-L1All-mask native5003.2K0.3325
PD-L1All-mask native5006.4K0.6701
IFNA2All-mask native5003.2K0.4711
IFNA2All-mask native5006.4K0.7145

These experiments test whether reward training based on masked infilling can achieve performance comparable to direct full-context editing. Although Masked GRPO improves native diffusion generation, it remains substantially below ProEdit at 6.4K updates on both PD-L1 (0.6701 vs. 0.9787) and IFNA2 (0.7145 vs. 0.9434). This supports directly training the parent-conditioned editing policy rather than relying on masked-infilling training followed by iterative denoising.

Reviewer_3Wu3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ProEdit uses a diffusion-pretrained DPLM-2 model to edit existing protein sequences in one step. Given a complete parent sequence, it samples all output residues in parallel and learns from structural rewards using parent-wise GRPO. The factorized policy allows exact calculation of action likelihoods and the conditional KL to a frozen reference. Experiments cover binder optimization against PD-L1 and IFNA2, with comparisons to native diffusion-rollout training and reward-weighted supervised training. The paper also examines raw proposal quality, scores from other structure predictors, sequence diversity, training cost, and editing designs from other generators.

Critical strengths
  1. Reward training improves the one-step editor. At 6.4K updates, mean iPTM of the selected test outputs reaches 0.883 on PD-L1 and 0.871 on IFNA2, compared with 0.238 and 0.242 for the frozen editor and 0.839 and 0.789 for reward-weighted supervised training (Table 1). GRPO and RW-SFT use the same initialization, parent conditioning, candidate collection, reward, and reference regularization. Best-of-eight selection adds only 0.013 and 0.011 reward over the raw proposal mean, with no parent fallback (Section 4.3). Most of the reward improvement is therefore already present in the proposals before selection.

  2. Training is much faster than the reported full-rollout baseline, and the quality gains also appear under other structure predictors. At a matched budget of 204.8K reward evaluations and 3.2K optimizer updates, training takes approximately 27.4 H200 GPU-hours, averaged across the two targets, compared with 738.3 for the native 500-step rollout baseline (Section 4.3). The AF-M-selected sequences also achieve higher iPTM than the other methods in Table 2 under Protenix-v1 and Boltz-2. These sequences are fixed before rescoring, with no reselection under either predictor. The gains therefore extend beyond the particular AF-M configuration used for training.

Critical weaknesses
  1. The main comparison gives ProEdit information that FullRollout-GRPO does not receive. ProEdit takes the complete parent sequence, whereas FullRollout-GRPO starts from an all-mask sequence and receives only the matched binder length (Table 8 and Appendix D.2). Table 3 shows that natural-parent context has a large effect on ProEdit’s performance. Matching reward evaluations and optimizer updates does not isolate the effect of one-step editing because the two methods receive different input information. The reported advantage therefore cannot be attributed to replacing iterative denoising with a one-step action alone. A comparison with iterative editing of the same parents is needed to assess this central claim. The approximately 27-fold speedup is also measured against the 500-step baseline, without a comparison to a shorter iterative schedule.

  2. The final checkpoint trades sequence-family diversity for higher reward. The effective number of 90%-containment families decreases from 200 at 3.2K updates to 73.26 on PD-L1 and 20.86 on IFNA2 at 6.4K updates (Table 4). The earlier checkpoint already achieves strong rewards with broad family coverage, so this trade-off alone does not show a failure of the editing approach. The random-sequence and all-mask controls show that input choice affects performance, but provide little information about how individual parent features guide the edits. Aggregate parent–output comparisons would help characterize this behavior beyond the examples in Figures 6–7.

Questions for authors
  1. Can the authors compare ProEdit with parent-conditioned iterative diffusion editing under the same reward-evaluation budget? The iterative control could start by partially masking the same parent sequences, with the corruption level and decoding length specified. Including a shorter decoding schedule would help assess how much of the reported quality and compute advantage depends on the 500-step baseline.

  2. Can the authors report aggregate parent–output sequence identity and structural similarity at the 3.2K and 6.4K checkpoints, using unrelated length-matched parents as a reference? Figures 6–7 provide individual examples; statistics across the test set would help explain how higher reward and lower family diversity relate to changes in the starting designs.

7 comments

Thank you for your careful review and for recognizing the strong reward and iPTM improvements over the frozen model and reward-weighted SFT, the quality of our raw policy proposals, and the computational efficiency of ProEdit. We also appreciate your positive assessment of the independent structure-predictor results. To address your concerns, we conducted additional parent-conditioned iterative comparisons and aggregate parent–output analyses, as detailed below.

1. Parent-conditioned iterative baselines (W1, Q1)

500 step and 64 step baselines

We trained three parent-conditioned GRPO baselines. M64 starts from a partially masked parent and iteratively completes the masked positions while keeping observed residues fixed. P64 and P500 use the complete parent as a fixed prefix and recover a fully masked part through 64 and 500 native denoising steps, respectively, which providing direct parent-conditioned rollout comparisons with ProEdit.

All methods used the same training budget of 3.2K optimizer updates and 204.8K training reward evaluations. Sampling and evaluation followed the protocol in Table 1, except that parent fallback is disabled.

MethodPD-L1 RewardIFNA2 RewardPD-L1 Neff,90N_{\mathrm{eff},90}IFNA2 Neff,90N_{\mathrm{eff},90}Training GPU-hours (PD-L1 / IFNA2)
ProEdit0.93320.8719200.00200.0021.6 / 33.1
M640.25290.2376200.00200.00110.0 / 120.6
P640.25900.35031.043.58165.3 / 175.1
P5000.24380.32442.915.061,303 / 1,321

All three parent-conditioned iterative baselines still trail ProEdit in reward. M64 retains high sequence-family diversity but achieves low reward, whereas P64 and P500 exhibit substantial family concentration. Increasing the number of denoising steps from 64 to 500 does not improve reward, despite substantially higher training cost. Under the same training budget, ProEdit achieves higher reward while maintaining (Neff,90=200N_{\mathrm{eff},90}=200) on both targets, with considerably fewer GPU-hours.

Additional baseline with masked infilling

We also trained a GRPO baseline with masked infilling (Masked GRPO). During training, we randomly mask positions in the parent sequence, keep visible residues fixed, and predict the masked positions in a single forward pass. This follows the masked-corruption and recovery formulation of diffusion pretraining, with predictions restricted to the masked positions.

The table reports inference results using the native 500-step diffusion decoder starting from an all-mask input.

TargetInference samplingOptimizer updatesReward
PD-L1All-mask native5003.2K0.3325
PD-L1All-mask native5006.4K0.6701
IFNA2All-mask native5003.2K0.4711
IFNA2All-mask native5006.4K0.7145

These experiments test whether reward training based on masked infilling can achieve performance comparable to direct full-context editing. Although Masked GRPO improves native diffusion generation, it remains substantially below ProEdit at 6.4K updates on both PD-L1 (0.6701 vs. 0.9787) and IFNA2 (0.7145 vs. 0.9434). This supports directly training the parent-conditioned editing policy rather than relying on masked-infilling training followed by iterative denoising.

2. Edit magnitude and output distribution for the same inputs (W2, Q2)

Similarity to the original parent sequence

Thank you for suggesting aggregate statistics. We analyzed all 200 held-out test parents per target for PD-L1 and IFNA2 at the 3.2K and 6.4K checkpoints. We measured sequence identity and binder-only TM-score between the edited proposals and their original parent sequences to assess whether changes in reward and sequence-family diversity accompanied differences in the extent of parent redesign.

TargetOptimizer updatesParent–output sequence identityBinder-only TM-scoreRewardNeff90
PD-L13.2K17.62%0.47510.9332200.00
PD-L16.4K16.91%0.47530.978773.26
IFNA23.2K12.69%0.39510.8719200.00
IFNA26.4K13.74%0.40940.943420.86

The results show ProEdit makes extensive sequence edits to the original parents, with similar average edit magnitudes at both checkpoints. Mean sequence identity changes from 17.62% to 16.91% for PD-L1 and from 12.69% to 13.74% for IFNA2. Mean binder-only TM-score changes from 0.4751 to 0.4753 and from 0.3951 to 0.4094, respectively.

Output distributions across checkpoints

Meanwhile, Reward increases and sequence-family diversity decreases, suggesting that average edit magnitude alone does not fully characterize how the policy evolves during training. We therefore further investigate whether the 6.4K policy merely concentrates probability mass on high-reward modes already well sampled at 3.2K. Using 1,600 unfiltered proposals per checkpoint, we compared their distributions in the frozen ESM-2 feature space.

  • Maximum Mean Discrepancy (MMD) [1]. Cross-checkpoint MMD reaches 0.648 on PD-L1 and 0.620 on IFNA2, compared with 0.010–0.023 for within-checkpoint splits, indicating a substantial shift in the proposal distribution.
  • Improved Precision (from Improved Precision and Recall, IPR) [2,4]. Only 1.0% of PD-L1 and 2.0% of IFNA2 proposals at 6.4K fall within the empirical feature regions defined by 3.2K samples, suggesting that later proposals are largely outside regions well covered by the earlier checkpoint.
  • PRD-based shared-bin overlap[3]. Using 20 shared feature bins, the overlap is 1.4%–1.9% on PD-L1 and 3.6%–5.4% on IFNA2, indicating substantial redistribution of probability mass across sequence regions.

These results suggest that the changes in Reward and sequence-family diversity are not driven by a shift in the policy's preference for editing more or fewer residues. Instead, the raw proposal distribution shifts toward regions of the ESM-2 feature space that were poorly covered by the 3.2K proposals, indicating that later training changes which edited sequences the policy favors.

References

[1] A Kernel Two-Sample Test · [2] Improved Precision and Recall Metric for Assessing Generative Models · [3] Assessing Generative Models via Precision and Recall · [4] Exploring Precision and Recall to assess the quality and diversity of LLMs

Reviewer_3Wu3OpenReview ↗

Official Comment by Reviewer 3Wu3

Thank you for adding the parent-conditioned baselines and the parent–output statistics. I have several additional questions so far. What masking rate and position-selection scheme were used for M64, and how was the parent prefix implemented in P64/P500? Could you report the requested sequence identity and TM-score against unrelated, length-matched parents, to show whether outputs are more similar to their own parents than to other parents? As moving to new sequence regions and concentrating into fewer families can happen together, how do the MMD/IPR/PRD results distinguish these effects and explain the observed diversity loss?

Reviewer_hmepOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ProEdit, a one-step protein editing framework that repurposes a diffusion-pretrained protein language model for reward-guided sequence optimization. Instead of following the original iterative denoising process, ProEdit conditions on a complete parent sequence and independently samples amino acids at all positions in a single forward pass. The resulting factorized policy enables tractable sequence likelihoods and exact reference KL computation, allowing the authors to apply GRPO directly using structural rewards from AlphaFold-Multimer. Experiments on PD-L1 and IFNA2 binder optimization show substantial improvements over diffusion-rollout GRPO and reward-weighted supervised fine-tuning, with significantly lower computational costs.

Critical strengths
  1. The paper raises an interesting question about whether the iterative generation process used during diffusion pretraining is necessary for downstream optimization. The motivation is clear and the proposed framework is computationally efficient.

  2. The empirical improvements are substantial, particularly compared with the native diffusion-rollout GRPO baseline. The authors also provide useful analyses of proposal quality, sequence diversity, parent conditioning, and generalization across structure predictors.

  3. The method is simple and easy to implement, with tractable likelihoods and reference KL regularization.

Critical weaknesses
  1. Limited expressivness of the factorized policy. Independently sampling amino acids seems restrictive for protein design, where coordinated mutations are often important. Although a factorized policy can concentrate on any deterministic optimum, this does not justify its ability to approximate a multimodal KL-regularized distribution with correlated residues. The practical impact of this restriction remains unclear.

  2. Insufficient understanding of one-step optimization. It is unclear why a single independent sampling step works so well for complex protein optimization. While the authors explore reward-accepted repeated editing, this introduces coordination through external iterations rather than within a single policy action. More analysis is needed to understand when one-step editing is sufficient and when iterative refinement is necessary.

  3. Lack of a fully controlled comparision. FullRollout-GRPO starts from an all-mask sequence, while ProEdit uses the complete parent. Interestingly, Table 3 shows that replacing the parent with an all-mask input substantially reduces ProEdit's performance, suggesting that parent conditioning is an important factor. A parent-conditioned iterative diffusion baseline is needed to isolate the contribution of one-step optimization.

  4. Limited analysis of residue dependecies. Although the paper provides sequence-family diversity statistics, it does not directly evaluate whether the learned policy preserves coordinated residue interactions. The reduction in diversity after further training to 6.4K updates also suggests a quality-diversity trade-off that deserves more investigation. The evaluation is further limited to computational predictions on two protein targets.

Questions for authors
  1. Why should one-step independent sampling be sufficient when successful protein optimization requires coordinated mutations across multiple residues?

  2. How would ProEdit compare with a parent-conditioned iterative diffusion editing policy under matched training budgets?

  3. Does the improvement mainly come from concentrating probability mass on a small number of high-reward sequence families?

  4. Could the authors investigate the performance under controlled higher-order residue interactions to better understand the limitations of the factorized policy?

7 comments

Thank you for recognizing the motivation, results, and computational efficiency of our work. We performed additional experiments with parent-conditioned iterative baselines, a gradient analysis of residue interactions under sequence-level reward, and diagnostics of the proposal distribution and local residue interactions.

1. Parent-conditioned iterative baselines (W3, Q2)

500 step and 64 step baselines

We trained three parent-conditioned GRPO baselines. M64 starts from a partially masked parent and iteratively completes the masked positions while keeping observed residues fixed. P64 and P500 use the complete parent as a fixed prefix and recover a fully masked part through 64 and 500 native denoising steps, respectively, which providing direct parent-conditioned rollout comparisons with ProEdit.

All methods used the same training budget of 3.2K optimizer updates and 204.8K training reward evaluations. Sampling and evaluation followed the protocol in Table 1, except that parent fallback is disabled.

MethodPD-L1 RewardIFNA2 RewardPD-L1 Neff,90N_{\mathrm{eff},90}IFNA2 Neff,90N_{\mathrm{eff},90}Training GPU-hours (PD-L1 / IFNA2)
ProEdit0.93320.8719200.00200.0021.6 / 33.1
M640.25290.2376200.00200.00110.0 / 120.6
P640.25900.35031.043.58165.3 / 175.1
P5000.24380.32442.915.061,303 / 1,321

All three parent-conditioned iterative baselines still trail ProEdit in reward. M64 retains high sequence-family diversity but achieves low reward, whereas P64 and P500 exhibit substantial family concentration. Increasing the number of denoising steps from 64 to 500 does not improve reward, despite substantially higher training cost. Under the same training budget, ProEdit achieves higher reward while maintaining (Neff,90=200N_{\mathrm{eff},90}=200) on both targets, with considerably fewer GPU-hours.

Parent-context controls in Table 3

Thank you for pointing out the performance differences across input conditions in Table 3. We designed this ablation to investigate whether the model leverages the parent context or ignores the input and samples unconditionally from a target-specific distribution of high-reward sequences.

Replacing the natural parent by a length-matched random sequence or an all-mask input substantially reduces reward, indicating that effective editing depends on the natural parent's context. This is consistent with ProEdit's training objective, which directly rewards sequence edits conditioned on the complete parent sequence. The objective does not train the model to reconstruct a sequence from an all-mask input.

Additional baseline with masked infilling

We also trained a GRPO baseline with masked infilling (Masked GRPO). During training, we randomly masked positions in the parent sequence, kept visible residues fixed, and predicted the masked positions in a single forward pass. This followed the masked-corruption and recovery formulation of diffusion pretraining, with predictions restricted to the masked positions.

The table reports inference results using the native 500-step diffusion decoder starting from an all-mask input.

TargetInference samplingOptimizer updatesReward
PD-L1All-mask native5003.2K0.3325
PD-L1All-mask native5006.4K0.6701
IFNA2All-mask native5003.2K0.4711
IFNA2All-mask native5006.4K0.7145

These experiments test whether reward training based on masked infilling can achieve performance comparable to direct full-context editing. Although Masked GRPO improves native diffusion generation, it remains substantially below ProEdit at 6.4K updates on both PD-L1 (0.6701 vs. 0.9787) and IFNA2 (0.7145 vs. 0.9434). This supports directly training the parent-conditioned editing policy rather than relying on masked-infilling training followed by iterative denoising.

2. Coordinated edits under factorized sampling (W1, Q1)

Factorized sampling does not prevent ProEdit from learning coordinated residue edits. Although residue positions are sampled independently conditioned on the parent sequence, the policy is optimized using the reward of the complete edited sequence, which captures the joint effects of mutations across positions.

This follows directly from the gradient of the expected reward. For a fixed parent, let

\qquad p_j=\operatorname{softmax}(z_j),$$ Holding the other positions' logits fixed gives $$\boxed{ \frac{\partial J_x}{\partial z_j(a)} =p_j(a\mid x) \left(\mathbb E[R(Y)\mid Y_j=a,x]-J_x\right). }$$ The conditional expectation averages over the other positions when position $j$ takes value $a$. This captures their joint effects on reward, including nonadditive interactions. For a simple illustration, consider two binary positions on a fixed background. Let $p,q$ be their mutation probabilities and define $$\Delta=R_{11}-R_{10}-R_{01}+R_{00}.$$ Then $$J_x=R_{00}+p(R_{10}-R_{00})+q(R_{01}-R_{00})+pq\Delta, \qquad \frac{\partial J_x}{\partial p} =(R_{10}-R_{00})+q\Delta.$$ **The nonadditive interaction term directly affects the optimization direction.** Even if a mutation is individually unfavorable, a sufficiently high probability of the other mutation and a sufficiently large positive interaction can make increasing its probability improve the expected reward. Thus, independent sampling does not imply that residue contributions are optimized independently. Learning coordinated residue edits is distinct from representing correlated multimodal distributions. As characterized by the product-family gap in our paper, a factorized policy may not fully represent the reward-regularized target distribution when residue choices are correlated. Our objective is to learn effective coordinated edits from sequence-level reward, rather than to match the full target distribution. We agree that allowing dependencies between sampled residues may further improve exploration of rare synergistic combinations and coverage of distinct high-reward modes.

3. Direct training of the one-step editing action (W2)

We directly train a policy over complete-sequence editing actions, rather than optimizing through the pretrained model's iterative denoising process. Given a complete parent sequence, the policy samples an edited sequence in a single parallel step and is updated using its sequence-level reward.

Theoretical basis for one-step editing

Two results directly motivate this design.

First, Proposition 1 characterizes the ideal reward-regularized objective over terminal sequence distributions. For a finite sequence space, finite rewards, a strictly positive reference policy, and λ>0\lambda>0, the objective

=\mathbb E_q[R(Y)] -\lambda D_{\mathrm{KL}}(q\Vert\pi_{\mathrm{ref}}(\cdot\mid x))$$ Its optimal terminal distribution is $$q^*\_{x,\lambda}(y) =\frac{\pi_{\mathrm{ref}}(y\mid x)\exp(R(y)/\lambda)}{Z_{x,\lambda}}.$$ This shows that the downstream objective can be defined directly over complete sequences without incorporating the pretrained model's native denoising trajectory. ProEdit further adopts a tractable, parallelizable terminal-sequence policy. Second, Proposition 2 establishes a local correspondence between the practical tokenwise actor-loss gradient and the complete-sequence policy gradient. Averaging over the G candidates from the same parent, with sampled candidates and detached advantages held fixed, gives $$\left.\nabla_\theta\mathcal L_{\mathrm{actor},x}\right|_{\theta_{\mathrm{old}}} =-\frac{1}{G L_x} \sum_{g=1}^G A_g \left.\nabla_\theta\log\pi_\theta(y_g\mid x)\right|_{\theta_{\mathrm{old}}}.$$ Thus, the tokenwise actor-loss gradient locally matches the advantage-weighted gradient of the complete-sequence policy, connecting the training objective to the one-step editing action used at inference. The factorized policy also enables exact computation of sequence-level action likelihoods and the conditional KL to the frozen reference policy. ### Raw policy proposals **Substantial reward improvements are already observed in unfiltered one-step proposals.** We evaluated all eight raw proposals from each of the same 200 held-out parents, without best-of-eight selection or parent fallback. The mean rewards are shown here: | Target | Frozen DPLM-2 | 3.2K updates | 6.4K updates | | --- | ---: | ---: | ---: | | PD-L1 | 0.11649 | 0.90715 | 0.96582 | | IFNA2 | 0.05567 | 0.83326 | 0.93231 | These results show that the trained policy produces high-reward proposals in a single forward pass, without relying on reward-based selection. ### When further rounds help We also evaluated reward-accepted repeated editing on candidates from different generators. For example, the mean reward of BindCraft candidates increased from 0.4617 to 0.8011 after one round and reached 0.9500 after eight rounds. Thus, a single editing round provides substantial improvement, while further rounds can yield additional gains depending on the input source. The same trained policy is reused across rounds without requiring a separate iterative editor.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.