acceptodds
Under review as a conference paper at ICLR 2027

ToxRedTeam: Red-Teaming Protein Toxicity Detectors with Structure-Aware Adversarial Attacks

Abstract

Toxic protein detection plays a critical role in screening DNA synthesis orders. Existing methods generally fall into two categories: homology-based detection and machine learning (ML)-based detection. Prior work has shown that homology-based approaches can be bypassed through strategically designed mutations to toxic proteins. In contrast, ML-based detectors learn sequence representations rather than relying on explicit matching, but their robustness to adversarially modified toxic protein variants remains largely unexplored. To address this gap, we propose ToxRedTeam, a systematic red-teaming framework for auditing protein toxicity detectors across both categories. Given query access to a detector’s prediction API, ToxRedTeam iteratively generates variants of a toxic protein using a masked protein language model, with the objective of inducing misclassification as non-toxic while preserving the protein’s toxic functionality. We empirically demonstrate the effectiveness of ToxRedTeam across five detectors, where it consistently outperforms baseline attack methods. Our code will be released upon publication.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI red teaming, Adversarial protein design, Biosecurity

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qjXZOpenReview ↗
Rating
2(+1)
Confidence
3(−1)
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This paper presents ToxRedTeam, a black-box red-teaming framework for protein toxicity detectors that iteratively mutates known toxic proteins while attempting to preserve their predicted structure. ToxRedTeam combines masked protein language models (ESM2 or SaProt), an ESMFold-based structural verifier using pLDDT and TM-score constraints, and greedy detector-feedback optimization with First-Choice Early Exit. However, the core technical ingredients are largely established component such as masked-PLM substitution, structure prediction/filtering, and first-improvement black-box search, so the novelty lies mainly in combining them for toxicity-detector auditing.

Critical strengths

The paper succeeds in making the biosecurity motivation concrete and in testing several heterogeneous detector families. The introduction clearly explains why simply producing a classifier-fooling sequence is insufficient in this domain which is useful and the threat model is relatively easy to understand.

Critical weaknesses

First, the paper does not isolate what actually causes the large attack gains. The main comparison is against ProteinMPNN, but ToxRedTeam changes both the candidate generator and the search procedure, most importantly adding repeated detector-feedback optimization. There is no matched ESM2/SaProt baseline that generates the same candidates without iterative detector feedback, nor a version that removes First-Choice Early Exit or the adaptive search logic. This is particularly problematic because the paper explicitly concludes about iterative use of detector feedback substantially amplifies attack success, yet the reported experiments never isolate that factor. The current results show that the full ToxRedTeam pipeline beats ProteinMPNN, but not which proposed component is responsible.

Second, there are concrete specification and reporting errors that make the experimental results difficult to trust. In Algorithm 1, an accepted improvement resets the stagnation counter to zero, but after breaking out of the candidate loop the pseudocode unconditionally increments the same counter under “No improvement,” contradicting the textual description of the search. More seriously, Section 5.5 states that 4,073 improvements among 55,827 queried candidates correspond to 4.2%, but this ratio is approximately 7.3%; 4.2% is obtained only when dividing by all 96,071 generated candidates. The query-efficiency reporting is also unnecessarily confusing: Table 2 reports medians over successful attacks, whereas the patience appendix reports a median over all targets, without making this distinction clear in the main text.

Questions for authors

(1) Which description corresponds to the actual implementation of Algorithm 1? (2) Can the authors confirm the denominator used for the reported 4.2% improvement rate in Section 5.5? (3) Can the authors provide a matched ESM2/SaProt baseline without iterative detector feedback?

2 comments
AuthorsOpenReview ↗

Response to Reviewer qjXZ

[Q1] Which description matches the implementation, Algorithm 1 or §4.4?

§4.4. Algorithm 1 is a transcription error and no reported number is affected — in the code the stagnation counter is incremented strictly inside the no-improvement branch, so an accepted improvement leaves c = 0, k = 1 as §4.4 states, and line 30 belongs inside that branch. The results come from the implementation, not the pseudocode.

Auditing the rest of Algorithm 1 turned up a second error we should report ourselves: line 13 (Fast-Escalation) is printed with a clamp, k ← min(k + 1, K_max), which §4.4 step 5 repeats and the implementation does not apply. The clamp is not equivalent — an all-rejected round spends no query and resets c ← 0, so under it neither loop guard can be reached on that path — and it is the unclamped version that produces the k_max_exhausted failure mode in Table 5. Both are corrected, along with two parameters we omitted: k_eff = min(k, K_max, L − |M|) and top_k_per_position = 5, the latter meaning a k = 1 round yields at most 5 candidates, not N_sample = 10.

On whether these errors reach the results: Appendix D's failure-mode table regenerates exactly from the run logs — 46/4/0, 45/5/0, 39/11/0, 36/6/8 and 23/27/0 for the five ESM2 runs — and the presence of k_max_exhausted in that table is itself what rules out the printed clamp, since under a clamp k can never exceed K_max and that termination reason could never fire. The pseudocode line was wrong; the procedure that produced the reported numbers is the one §4.4 describes.

[Q2] Can you confirm the denominator for the 4.2% improvement rate in §5.5?

The denominator is the 96,071 generated candidates: 4.2% is 4,073/96,071. Over the 55,827 candidates actually queried the rate is 7.3%, and the 96,071 count precedes the N_sample truncation, so 4.2% is a rate per candidate proposed rather than per query spent. The revised §5.5 reports both with their denominators named.

[Q3 / W1] Can you provide a matched ESM2/SaProt baseline without iterative detector feedback?

We have removed the claim this control was needed to support, rather than offer a substitute number. Our comparison varies the proposal model and the search procedure together, so it cannot attribute the gain to feedback; the §6 sentence asserting that iterative feedback substantially amplifies attack success is deleted, and we will not reassert it without the matched baseline you describe.

[W2 / Presentation] The contribution list now leads with the audit and the evaluation protocol, and the search algorithm is no longer offered as a methodological contribution — the components are individually established, and what the paper claims is the audit they enable, not the parts. Beyond the two Algorithm 1 errors and the §5.5 denominator, we also corrected: the two omitted parameters above; the unlabelled medians in Table 2 (over successful attacks) and Appendix C (over all 50 targets); the Introduction at L060, which said "substitutions at conserved positions" and contradicted §4.2, where the top 25% most conserved are frozen; and the L042 punctuation. If you can point to sections that were hardest to follow, we will restructure them.

Thanks for the authors’ clarification. I think the response clarifies the algorithm and consistency issues well, and the scoping of the claims is appropriate. However, I still believe this paper needs more experiments, especially more ablations, to show where the attack gains actually come from. I will raise my rating to 2.

Reviewer_CcTNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

Prior work showed that homology-based biosecurity screening can be cracked, and ML-based methods were proposed as a replacement. This paper tests whether those ML-based methods are more robust. This paper also proposes a new method, ToxRedTeam, which mutates toxins with a masked protein language model and tests every candidate through a protein structure generator as a verifier, then greedily searches with a given query budget. Under their experiment setup, the ESM-2-based model reaches 72-92% ASR on ML-based detectors and 46% on Commec.

Critical strengths

This work follows directly from a prior works' recommendation and tests the robustness of a category of works which nobody does before. The results matters for how screening gets to built. Restricting ASR to D_valid is the right design This rules out the pre-existing misses by attacking those toxins a detecter have been caught. The full D_valid runs also match the 50 target subsets, so the analysis is consistent. Appendix A itself is very helpful. It provides a list of workable detectors and also explains the choice of four detectors mentioned in the paper and saves the next group the effort.

Critical weaknesses
  1. Efficiency is claimed but not benchmarked. The second contribution emphasizes that this is a query-efficient greedy search. All the efficiency-related metrics are tested within ToxRedTeam, for example, median 63 queries, 77.2% under 200, etc. However, these pieces of evidence are not compared with the baseline because ProteinMPNN only gets one query. Therefore, extra controls are needed to prove the efficiency claim: for instance, random-substitution search under the same budget.
  2. V is not ablated, and the final-variant fidelity is not presented. Gates are run to compare with wild type at every acceptance, so naturally TM score is larger than 0.5. However, there is no clear statement of where exactly the TM score lies, and how the search drifts without V.
  3. Claimed that toxic functionality is preserved but never validated. The abstract states that the toxic functionality is preserved, but according to the appendix, there is no wet-lab verification. So this claim stays unproved.
  4. No seed variance. ToxinPred2's −6% is likely to be just noise, and ToxinPred3's +10% also needs further inspection to decide whether it is noise or actual change.
Questions for authors
  1. How does the ASR vary under the baseline using a random/no-early-stop algorithm under the same budget as ToxRedTeam? How many queries per success are needed by a no-early-stop version of ToxRedTeam?
  2. How does the ASR vary after removing V? What are the TM score and Δ\DeltapLDDT distributions for the full setup and the V-removed version?
  3. What are the per-target results for the masker comparison? How is the seed-to-seed ASR spread over a single detector?
3 comments
AuthorsOpenReview ↗

Response to Reviewer CcTN [1/3]

[Q1] ASR under a random baseline and a no-early-stop variant at matched budget; queries per success without early stopping.

No-early-stop. Because First-Choice Early Exit accepts the first improving candidate, a no-early-stop variant that accepts the same candidate follows an identical trajectory and differs only in querying every verified candidate — so the comparison holds everything else fixed. Table 3 already reports the magnitude: 33,857 candidates survived V but went unqueried, against 55,827 queried, so that variant would spend 89,684 queries for the same accepted variants. We do not report a no-early-stop ASR, which would need an actual run.

Random substitution. The claim this control bears on is withdrawn rather than defended: query efficiency is removed from the contributions and the comparative framing in §5.3 is withdrawn, so nothing we now claim requires PLM guidance to be the operative ingredient. The control itself we have not run, and the audit's conclusion does not depend on it: if random substitution at matched Q_max and under the same verifier reached our ASR, these five detectors would be evadable by fold-preserving perturbation generally — a broader screening failure than the one we report, not a narrower one. A matched-budget pass is also not cheap: detector queries account for 58% of our measured runtime, and CSM-Toxin's external API costs a measured 32.8 s per query — which is also why its budget is Q_max = 500 rather than 1000.

[Q2] ASR after removing the verifier; TM-score and ΔpLDDT distributions.

Every accepted variant satisfies TM-score ≥ 0.50 and ΔpLDDT ≥ −10.0 by construction — V is a hard gate, not a soft penalty. What the submission did not report is the distribution above those thresholds, a reporting omission rather than a compute limit; if the accepted variants sit close to the gates, their admissibility rests on the published thresholds themselves rather than on any margin above them, and we cannot assure you otherwise. On final-variant fidelity the revision now reports the sequence-level half of your question — new Table 6 in Appendix E, recovered exactly from the per-iteration logs with no new compute. For the ESM2 configuration that carries Tables 1, 3 and 5, over the successful targets whose per-iteration logs are retained:

Detector (ESM2)npositions mutated, med [min, max]identity to WT, med [min, max]
ToxinPred24623 [2, 111]0.858 [0.370, 0.992]
ToxinPred34524 [2, 42]0.788 [0.494, 0.987]
Toxify398 [1, 42]0.926 [0.795, 0.997]
CSM-Toxin295 [2, 25]0.972 [0.937, 0.993]
Commec2332 [1, 90]0.797 [0.433, 0.992]

The attack is not uniform: CSM-Toxin is evaded at a median of 5 substitutions, Commec needs 32. We report the low tail rather than the medians alone — the least conserved accepted variants sit at 37% identity, which is precisely the regime a structural gate rather than a sequence-similarity gate is there to govern, and the same regime as the AI-generated redesigns Wittmann et al. (Science 2025) screen for. The TM-score and ΔpLDDT values were not retained (only accept/reject counts), so those two distributions require re-folding the final variants and their wild types; that we have not done, and Appendix E now says so in those terms.

The verifier-removal ablation is not run. Its direction is predictable and we state it so the gap is not read in our favour: removing V should raise raw ASR and lower the admissible share, so raw ASR alone would flatter the method.

AuthorsOpenReview ↗

Response to Reviewer CcTN [2/3]

[Q3] Per-target masker results, and seed-to-seed ASR spread.

What seed repetition bears on is the comparison between maskers, and that comparison is withdrawn: Table 2 is now a practical comparison of two available proposal models, with no conclusion drawn about structural conditioning. We have no repeated seeds on any detector, so the ESM2–SaProt differences cannot be presented as effects. What it does not bear on is the audit's own conclusion: on each successful target the run produced a specific admissible variant the detector scored below τ_safe, and a constructed witness has no sampling distribution. Seeds and intervals govern how often; they do not reach whether. On feasibility: the attack runs behind Tables 1–3 total ≈870 h of single-stream compute (41% structural verification, 58% detector queries), so three seeds across five detectors is three further full passes of that size. The submission also reports only marginal counts, not the per-target pairing. What we can add is the uncertainty the submission omitted — Wilson 95% intervals on Table 1, now in §5.3:

DetectorBaselineToxRedTeam-ESM2ToxRedTeam-SaProt
ToxinPred234.0% [22.4, 47.8]92.0% [81.2, 96.8]86.0% [73.8, 93.0]
ToxinPred32.0% [0.4, 10.5]90.0% [78.6, 95.7]100.0% [92.9, 100.0]
Toxify52.0% [38.5, 65.2]78.0% [64.8, 87.2]78.0% [64.8, 87.2]
CSM-Toxin66.0% [52.2, 77.6]72.0% [58.3, 82.5]86.0% [73.8, 93.0]
Commec14.0% [7.0, 26.2]46.0% [33.0, 59.6]56.0% [42.3, 68.8]

Because only marginals are available, we bound the paired test rather than point-estimate it: with a, b the success counts out of 50, n₁₀ − n₀₁ = a − b and 0 ≤ n₀₁ ≤ min(b, n − a), and the exact McNemar p-value is monotone in n₀₁, so both endpoints bracket every admissible pairing. Under the least favourable one, ToxinPred2 p ≤ 1.1 × 10⁻⁶, ToxinPred3 p ≤ 1.3 × 10⁻¹², Commec p ≤ 5.2 × 10⁻³, Toxify p ≤ 4.1 × 10⁻², and CSM-Toxin p ∈ [0.25, 0.72]. We therefore withdraw the CSM-Toxin ESM2 margin (+6 points) — it is indistinguishable from zero under our own counts, and it is also the detector whose API returned different scores for identical inputs (now moved from Appendix E into §5.3). Toxify is only marginally significant in the worst case, which "52% → 78%" did not convey.

AuthorsOpenReview ↗

Response to Reviewer CcTN [3/3]

[W3] Toxic functionality claimed but never validated.

The standard of evidence here is the one this literature uses, and we are keeping it; what needed correcting was the wording, and it is corrected. The gates are not ours: TM-score ≥ 0.50 and ΔpLDDT ≥ −10.0 are the criteria Wittmann et al. (Science 2025) use to decide which screening evasions pose genuine concern, and §3 adopts both unchanged. That work frames its own redesigned variants as "synthetic homologs more likely to retain wild type-like function" — a predicted likelihood, not a validated property, and that is the register the submission should have used. The submission's Appendix E already stated that no wet-laboratory validation was performed and that structural plausibility rests on in-silico metrics, consistent with that work. Where the submission went further than its own gates was the abstract, which asserted retained function as a fact; it now claims only what V certifies, and §3 states explicitly that fold preservation is necessary but not sufficient for function.

We are not offering an assay in its place, and we do not think this line of work should be asked for one: synthesizing and testing novel toxin variants is the activity synthesis screening exists to prevent, and our release contains no synthesis-ready sequences or wet-lab protocols. The audit's conclusion also does not rest on the functional claim — screening must clear or flag an order before anything is known about the ordered protein's activity, so a detector that clears a predicted fold-preserving homolog of a known toxin has failed in its screening role whatever an assay would later show. And the residual uncertainty runs in the conservative direction: if fold retention overestimates functional retention, our ASRs overestimate real-world risk, so the audit is an upper bound on detector failure. We add no in-silico functional proxy, since a panel we have not computed would not be evidence either.

Reviewer_t6jYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ToxRedTeam, a framework combining protein language models, structural verification, and greedy search to evaluate the adversarial robustness of protein toxicity detectors. Besides,experiments in this work reveal substantial vulnerabilities to structurally constrained adversarial variants.

Critical strengths
  1. The paper addresses a important biosecurity problem by investigating the adversarial robustness of protein toxicity detectors.

  2. The evaluation covers five detectors with different architectures, and the integration of structural verification provides a reasonable approach to assessing biologically plausible adversarial variants.

Critical weaknesses
  1. The paper claims to preserve toxic functionality, but validation relies only on predicted structural confidence and fold similarity. Somehow, these metrics can't establish whether the generated variants retain their original biological activity.
  2. The baseline comparison is not sufficiently fair. ProteinMPNN uses only one detector query per target, whereas ToxRedTeam is allowed up to 500–1,000 queries.
  3. The methodological novelty is limited, as the framework primarily integrates existing protein language models, structural prediction, and greedy search. The paper also lacks ablations isolating the contributions of structural verification.
  4. The main experiments use only the first 50 valid targets per detector without explaining their representativeness, limiting the generalizability of the conclusions.
  5. Several implementation and reporting inconsistencies remain. Section 5.5 also incorrectly reports 4,073 improvements out of 55,827 queries as 4.2% instead of a correct percentage.
Questions for authors
  1. What evidence supports the claim that the generated adversarial variants preserve toxic functionality rather than merely structural similarity?
  2. How much of the improvement over ProteinMPNN comes from the substantially larger query budget? What are the independent contributions of structural verification?
  3. How were the 50 evaluation targets selected?
2 comments
AuthorsOpenReview ↗

Response to Reviewer t6jY [1/2]

[Q1 / W1] What evidence supports preserved toxic functionality rather than structural similarity?

The evidence is in-silico and structural: fold retention and predicted-confidence gates, at the thresholds Wittmann et al. (Science 2025) use for exactly this judgment, adopted unchanged in §3. That literature frames the outcome as a likelihood rather than a validated property — their own abstract reports improved detection of "synthetic homologs more likely to retain wild type-like function" — and that is the register the submission should have used. That is the standard of evidence in the screening-evasion literature, and the submission already said so — Appendix E states that no wet-laboratory validation was performed and that structural plausibility rests on in-silico metrics, consistent with that work. The abstract, however, stated this as a fact rather than as what the gates certify. That is what we have fixed: it now claims only what V admits, and §3 states that fold preservation is necessary but not sufficient for function.

We are not offering an assay in its place, and we do not think this line of work should be asked for one: synthesizing and testing novel toxin variants is the activity synthesis screening exists to prevent, and our release includes no synthesis-ready sequences or wet-lab protocols. The audit also does not rest on the functional claim — screening must clear or flag an order before anything is known about the ordered protein's activity, so a detector that clears a predicted fold-preserving homolog of a known toxin has failed in its screening role whatever an assay would later show. And the uncertainty runs in the conservative direction: if fold retention overestimates functional retention, our ASRs overestimate real-world risk, so the audit reads as an upper bound on detector failure.

[Q2 / W2] How much of the gain over ProteinMPNN is the larger query budget, and what does structural verification contribute?

We are withdrawing the comparative claim rather than defending it — a one-shot sampler with a single query cannot be compared with an iterative procedure at Q_max = 500–1000. "Substantially exceeding the ProteinMPNN baseline" and the "+6% to +88%" range are removed. We have neither the matched one-shot control nor a Q_max sweep. What survives does not depend on the budget: robustness to one-shot redesign does not imply robustness to adaptive querying.

On verification: V is not vacuous — it rejects 6,387 of 96,071 generated candidates (6.6%), from 16.9% on ToxinPred3 to 0.3% on Toxify (Table 3) — but a rejection rate is not a controlled removal, and the ablation that would isolate V is not run.

[Q3 / W4] How were the 50 evaluation targets selected, and are they representative?

The first 50 entries of each detector's D_valid in benchmark-file order, fixed before any attack runs. The target list is built by iterating the benchmark table in file order, D_valid filtering preserves it, and the cap is a deterministic prefix — no shuffling, sampling, sorting or seed in that path, and no attack outcome enters the choice. §5.1 now states this.

The submission also already contained the representativeness evidence and under-presented it: at full D_valid, ToxinPred2 gives 91.7% [87.4, 94.6] on 228 targets against 92.0% on its subset, and ToxinPred3 gives 94.2% [88.9, 97.0] on 137 against 90.0% — both inside the 50-target intervals, with ToxinPred3's full set higher. Passes on the other three are blocked by the interfaces (CSM-Toxin's rate-limited API, which is why its budget is 500; Commec's 30–60 s HMMER queries). Table 1 conflated two quantities as submitted: the revision splits |D_valid| and n_eval and states the evaluated scope explicitly.

AuthorsOpenReview ↗

Response to Reviewer t6jY [2/2]

[W3] Novelty and missing ablations. The contribution list is repositioned around the audit and the evaluation protocol: the components are individually established, and the claim is the audit they enable. The verifier-removal ablation is not run, and we have no repeated seeds, so no masker-level claim is made. The audit's conclusion does not rest on that attribution: component ablations would establish why the attack works, whereas the claim is that each of these detectors admits variants satisfying the structural constraints, and on every successful target that variant was constructed rather than estimated.

[W5] Implementation and reporting inconsistencies, including the 4.2% figure. All corrected: the 4.2% denominator (4,073/96,071 generated vs 7.3% of 55,827 queried, generated count labelled pre-truncation); Algorithm 1 line 30, attributed to the pseudocode rather than the implementation; a second pseudocode error at line 13 that no review raised and we report ourselves, the Fast-Escalation clamp the implementation does not apply; two omitted parameters, k_eff and top_k_per_position = 5; the unlabelled medians in Table 2 and Appendix C; and the Introduction's contradiction of §4.2 plus the L042 punctuation. That is four beyond the one you cite; if you can point to others we will correct them. On whether any of it reached the results: Appendix D's failure-mode table regenerates exactly from the run logs (46/4/0, 45/5/0, 39/11/0, 36/6/8, 23/27/0 for the five ESM2 runs), so the pseudocode and reporting errors were in the write-up, not in the procedure that produced the numbers.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common Response to All Reviewers

We thank the reviewers. Two of the recurring concerns are not of the same kind, and we separate them.

On toxic function (CcTN W3, t6jY W1): the evidentiary basis is the one Wittmann et al. (Science 2025) established for this problem — in-silico fold retention and predicted-confidence gates, at thresholds we adopt from them unchanged (§3) — and the submission already stated that no wet-laboratory validation was performed, consistent with that work (Appendix E). What needed correcting was the wording: the abstract stated retained function as a fact, and it now claims what the gates certify and no more.

On feedback amplification (qjXZ Q3, t6jY W2): that claim does lack its control, and we have deleted it rather than defend it.

Neither bears on what the audit actually constructed. On every successful target the run produced a specific variant that passed the structural gate and that the detector scored as non-toxic. That is a witness, not an estimate: confidence intervals and repeated seeds govern the rates, and we now report both honestly, but they do not reach the existence of the evasions themselves.

The revised PDF carries this through: both Algorithm 1 transcription errors and the §5.5 denominator are corrected, every rate in Table 1 now carries a Wilson 95% interval, the comparative margins those intervals do not support are withdrawn, the two omitted search parameters are stated, and a new Appendix F supplies the statistical methodology. Where a request bears on a claim the evidence does not carry, we have narrowed or removed the claim rather than argue for it. All changes are in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.