acceptodds
Under review as a conference paper at ICLR 2027

SafeProtein++: Cross-Architecture Red-Teaming and Unlearning of Protein-Generative Models

Abstract

Proteins execute nearly every cellular function, and neural networks now design them from a multi-modal context—a functional label, homology sequences or a backbone structure—denoising sequence and structure jointly or decoding one amino acid at a time. These models deliver functional enzymes, binders and antigens, but their susceptibility to jailbreaking is underexplored, raising the concern of harmful outputs such as toxins and viral proteins. We introduce SafeProtein++, the first framework to evaluate the jailbreak vulnerabilities of protein generative models across all three main architecture families and the safeguards proposed to close them. It provides (1) a cluster-disjoint benchmark of 854 proteins supplying identical cells to all three generator families, judged jointly on sequence and folded structure; (2) a pathogenic-guided beam search that steers chunked decoding toward high-hazard completions; and (3) a reinforcement-unlearning recipe that removes hazardous capability from the weights at affordable cost in benign generation. Empirically, masking conserved positions is the strongest attack for every family, and unlearning cuts hazardous attack success to a few percent on every generator, at a measurable cost in structural fidelity. The guided beam lifts attack success on every mask condition, yet against an unlearned model it collapses to a fraction of the base rate: search-time guidance re-ranks the victim’s own distribution rather than restoring removed hazard. A case study on amylin reveals a hazard invisible to every guardrail: completing its masked amyloid core raises predicted aggregation propensity above the native hormone. Known-target reconstruction is therefore insufficient for safety certification; despite unlearning reduces covered hazards, while open-set coverage remains unresolved. Code and data will be released publicly upon paper acceptance, and an anonymized version is available at https://anonymous.4open.science/r/ICLR-2027-SAFEPROTEIN-PP/README.md.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein Generative Models, Red-teaming, Reinforcement Unlearning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XiSXOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies attacks against protein generation models from three families (AR, masked diffusion, inverse folding) meant to generate hazardous proteins. The attack pipeline is to start at a known hazardous protein, then apply some amount of masking following a particular strategy, and use the protein generation model to fill in the masks. The paper shows that one way to strengthen the attack for autoregressive models is to use guided beam search where we rank candidates using a hazard classifier. On the other hand, it shows that a consistent way to weaken the attacks is to first use RL to "unlearn" the hazardous proteins with either NPO or GRPO.

Critical strengths

The authors consider various families of protein generation models, not just autoregressive or masked diffusion models.

Critical weaknesses
  • I found the paper to be quite poorly written. For example, it was not clear what the main contributions were as the abstract, introduction, and Figure 1 all highlight different things. Moreover, a lot of dense jargon is introduced without explanation, and often not important for the main text (such as the guardrail latter which barely gets mentioned after the introduction).

  • The beam search attack protocol seems quite flawed, as the toxicity classifier was not used for chains longer than 50 residues.

Questions for authors

None

Reviewer_2YSUOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

SafeProtein++ studies hazardous-target reconstruction and unlearning in five protein-generative models spanning autoregressive, inverse-folding, and masked-diffusion architectures. The paper introduces a homology-cluster-disjoint benchmark, evaluates reconstruction using joint sequence and predicted-structure criteria, and compares two unlearning objectives, NPO and GRPO. It also evaluates guided decoding and presents an amylin case study to illustrate limitations of identity-based screening. The main contribution is a common evaluation framework and an empirical comparison of unlearning across different model families.

Critical strengths
  1. The evaluation includes several useful controls: homology-cluster-disjoint splitting, consistent structure prediction for references and completions, and cluster-aware uncertainty estimates for the main defense comparisons. Unattempted or undecidable cases are explicitly distinguished from successful defenses.
  2. Extending unlearning across three architecture families is a meaningful contribution, particularly the treatment of masked diffusion and the disclosure of differences between training and native inference schedules.
  3. The comparison between NPO and GRPO reveals an important safety–utility trade-off: stronger suppression of target reconstruction can coincide with reduced structural fidelity and benign-task performance.
  4. The authors explicitly disclose limitations of the proxy metric, adaptive evaluation coverage, and case-study execution. These disclosures make the experimental scope easier to assess.
Critical weaknesses
  1. The open-weight robustness claim is broader than the evaluation. Placing a safeguard in the weights makes it independent of a deployment wrapper, but does not establish that it remains effective after those weights are modified. The search-time evaluation includes ProtGPT3-GRPO and additional zymCTRL and ProteinMPNN checkpoints in Appendix A.4, although the ProteinMPNN NPO run is partial. This is broader coverage than the main-text example alone, but it does not assess robustness to subsequent model updates. Search can amplify residual capability, so these tests are useful, but they address a different question from the durability of protection under weight access. The conclusions should distinguish these settings and limit robustness claims to the configurations evaluated.

  2. The reconstruction benchmark does not directly establish the practical risk or comparative safeguard coverage. The main success criterion measures recovery of known hazardous targets using sequence and predicted-structure similarity. It does not demonstrate harmful function, screening evasion, or the generation of previously unrecognized hazards. Conversely, lower reconstruction success does not establish the absence of hazardous functionality. The guardrail ladder is a useful organizing framework, but the main quantitative comparisons primarily concern undefended models and weight-level unlearning, rather than a controlled comparison of all safeguard levels. The paper should separate these conclusions and clarify what generative models contribute beyond simpler reference methods under the information supplied by the benchmark.

  3. Several headline claims need to be aligned with the reported comparisons. The low residual ASR figure of ≤1.7% applies to NPO as a mean over six conditions, rather than a uniform bound for both NPO and GRPO. The objectives should be reported separately, with condition averages distinguished from worst-condition results. For guided decoding, the main ProtGPT3 experiment reports higher point estimates in all six conditions; Table 7 instead concerns the additional zymCTRL and ProteinMPNN experiments, where no individual paired sign test reaches p < 0.05. These observations do not contradict one another, but the abstract should distinguish point-estimate gains from statistically established improvements and specify their scope. Attribution is also limited by simultaneous changes to temperature, residue-selection rule, and budget, while the compute-matched example favoring repeated standard sampling limits a general efficiency claim. More fundamentally, Appendix A.4 reports that the hazard scorer is inactive throughout the ProteinMPNN runs and beyond its length limit in the autoregressive runs, leaving likelihood-based ranking in those steps. The additional results therefore do not establish a benefit from hazard-specific guidance; this distinction should be prominent in the main text.

  4. The case study supports a predictor-score shift more directly than increased biological hazard. The amylin analysis illustrates why similarity to a reference database is not a complete hazard measure. It also raises a coverage question for the defense: to what extent does unlearning generalize to hazards not represented in its training set? A non-hit against the hazard database does not by itself establish the absence of relevant training coverage, and the current case study does not demonstrate either protection or failure of protection by an unlearned checkpoint on this hazard. The reported change in aggregation-related predictor score also does not by itself establish increased toxicity or functional hazard. The two reference controls provide limited calibration for interpreting the magnitude of this shift. The case study should remain explicitly exploratory, and the inactive structure-dependent scoring components should be stated alongside its results. Claims about stronger hazard require evidence that the readout tracks the relevant biological endpoint.

  5. The safety–utility trade-off deserves greater emphasis. For ProteinMPNN, benign reconstruction success falls from 38.6% to 9.6% under NPO at [email protected] (Tables 2 and 10). This is a substantial loss on the specified reconstruction task, although it does not directly quantify overall protein-design utility. Together with the reductions in structural fidelity, these observations are central to assessing whether the resulting models remain useful. They should be presented prominently and distinguished from evidence of selective removal of hazardous capability. The current experiments demonstrate a trade-off at the tested operating points; they do not establish an unavoidable lower bound on utility loss.

Questions for authors
  1. Can the authors distinguish wrapper independence, robustness to search-time adaptation, and robustness to subsequent weight modification, and state which claims are supported by the current experiments?
  2. Can NPO and GRPO results be summarized separately, including condition means, worst-condition ASR, and corresponding benign-utility retention?
  3. What evidence supports the durability of the safeguards under subsequent weight modification, beyond the search-time evaluations reported here?
  4. Can the guided-decoding claims distinguish the main ProtGPT3 point-estimate gains from the additional paired tests, and separate hazard-specific guidance from ordinary search effects under controlled settings and computational budget?
  5. How do simple non-neural reference methods perform under the same benchmark information constraints, and what additional capability can consequently be attributed to the generators?
  6. What evidence calibrates the case-study predictor against the biological endpoint of interest, and how should its score change be interpreted given the limited reference controls and inactive scoring components?
Reviewer_jrfuOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SafeProtein++, a red-teaming study and also a defense study of jailbreaks of protein generative models. The study covers three architecture families and benchmarks with 854 proteins. In addition, a chunked beam search re-ranked by a pathogenicity scorer, and removal of hazardous capability from the weights by NPO with a KL anchor and by GRPO with structure-aware rewards, are further introduced as unlearning exploration.

Critical strengths
  1. The framework is well-designed. It covers autoregressive, inverse-folding and masked-diffusion models under a single pipeline. And studying satefy in bio AI models is meaningful.

  2. The experiments are thorough. Five generators and two defended checkpoints are explored across all six mask × ratio conditions. It provides extensive analysis and benchmarking of the safety issue across different types of protein foundation models under different conditions. Both red-teaming and unlearning are covered, making it a comprehensive study.

Critical weaknesses

In this paper, the reward, i.e., Eq. (4) and Eq. (5), and the judge both use RMSD and retrieval to determine "hazard". However, dissimilarity to a specific target does not mean that the output leads to no hazardous effect. It could be dissimilar in structure but similar in function.

Questions for authors
  1. Could the authors give a formal definition of the safety issue this paper studies? My understanding is that a protein is hazardous only when it is actually produced. In my view, a computational output that cannot cheat the judge cannot be treated as hazardous, because it leads to no hazardous effect. In this paper, it seems that "hazardous" means "close to something already on file", which is a property of retrieval rather than of harm. Besides, sentences like "raising the concern of harmful outputs such as toxins and viral proteins" does not make sense to me, as toxins and viral proteins output by models have no harmful effect if they can be easily identified. I think this paper needs a clear scope statement, rather than directly borrowing the words "safety" and "jailbreak" from language models, where the output itself is the thing that can cause bad effect.

  2. What if the defended model is further fine-tuned?

Comments on the paper

1 comment

I would like to thank the authors for their rebuttal. Dear Reviewers, as we are approaching the end of the discussion period, I would appreciate it if you could take a moment to acknowledge the rebuttal and see if it addresses your concerns or no. Thank you for your time and service.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.