acceptodds
Under review as a conference paper at ICLR 2027

The Effectiveness and Durability of Pretraining Data Filtering in Protein Language Models

Abstract

Pretraining filtering is one of the most common safeguards for open-weight language models. Here we study its effectiveness, durability and scalability using the ESM-2 family of protein language models. We find that filtering viral proteins from the pretraining data meaningfully degrades masked language modelling, mutation effect prediction and long-range contact prediction on viral proteins, while largely preserving performance on non-viral proteins. These effects are concentrated among proteins from human-infecting viruses, and grow as we scale models from 8M to 3B parameters. As models increase in size, we find that stringent filtering becomes more effective. However, fine-tuning on sequences from the removed viral families restores the lost performance. Although the cost to recover performance increases with model scale, even for the largest models it costs about 80 USD of H100 time. This work establishes pretraining filtering as moderately effective at selectively degrading performance on viral proteins, but not durable against fine-tuning with modest computational resources and access to relevant data.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
model safeguards, data filtering, protein language models, tamper resistance, mitigations

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Jb2VOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the effectiveness, durability, and scaling behavior of pretraining data filtering in protein language models. The authors train models under different levels of viral sequence exclusion based on ESM-2. They evaluate the impact of filtering on masked language modeling, mutation effect prediction, and contact prediction, and further examine whether the lost performance can be recovered through continued fine-tuning on the excluded sequences. The results suggest that filtering selectively degrades performance on viral proteins, with larger effects observed at greater model scales, while largely preserving performance on non-viral proteins.

Critical strengths
  1. The paper addresses a relevant question concerning the effectiveness and durability of pretraining data filtering in biological foundation models, extending previous investigations on general-purpose language models to protein language models.
  2. The analysis of different viral families is informative, particularly the observation that filtering effects vary substantially across protein families. The quantification of recovery costs also provides useful empirical evidence for evaluating the durability of pretraining filtering.
Critical weaknesses
  1. unclear biological impact of the observed filtering effects

A central motivation of the paper is to evaluate pretraining data filtering as a safeguard for biological foundation models. However, the experimental results reveal substantial heterogeneity across viral families, raising questions about the generality and practical significance of the reported filtering effectiveness. In particular, Figure 4 and Table 5 show that the degradation is heavily concentrated in the human-infecting viral shell, which accounts for 81.6% of the overall cross-entropy increase at 650M. Further analysis indicates that Retroviridae alone accounts for approximately 77.5% of the degradation at 3B among viral families with at least 20 held-out sequences. More importantly, the relationship between the measured performance degradation and the intended biosecurity impact remains insufficiently established. What's more, if the safeguard is investigated solely from the perspective of pretraining data filtering, how does it differ from conventional machine unlearning?

  1. limitation on evaluation of downstream capability suppression and recovery

The paper evaluates filtering and recovery using masked language modeling, zero-shot mutation effect prediction, and contact prediction through attention-based linear probes. While these evaluations provide useful measurements of the learned representations, they offer a limited characterization of the real model's functional capabilities. In particular, the recovery experiments rely on continued masked language modeling using the excluded viral sequences. Recovery is then assessed by comparing the resulting models against their unfiltered counterparts on the same evaluation metrics. The paper does not systematically examine whether the filtering effects persist after supervised adaptation to downstream biological tasks. Given the paper's motivation of evaluating pretraining data filtering as a safeguard, could the authors further investigate its effectiveness under downstream supervised fine-tuning (SFT)? In particular, it would be valuable to compare three settings:

  • (1) filtered pretraining followed by filtered SFT,
  • (2) filtered pretraining followed by unfiltered SFT,
  • and (3) unfiltered pretraining followed by unfiltered SFT

under comparable training budgets. How much do these settings differ in downstream predictive performance and generalization to unseen proteins or mutations? More importantly, does filtering at the pretraining stage lead to persistent changes in the model's learned capabilities, particularly its generalization ability, once downstream supervision is introduced? Such comparisons would provide stronger evidence for the effectiveness and durability of pretraining filtering as a safeguard.

  1. insufficient consideration of training budget in the scaling analysis

One of the paper's main findings is that filtering effectiveness increases with model scale. However, all models, ranging from 8M to 3B parameters, are pretrained using the same 150,000 optimization steps and approximately 315B training tokens. Although this provides a consistent training protocol, the degree of training sufficiency may vary substantially across model scales. Larger models generally require different training budgets to reach comparable levels of convergence. Also, the authors train for substantially fewer steps than the original ESM-2 models.

Questions for authors

All my main concerns are listed in the weakness section. I would appreciate it if the authors could provide further explaination and, where possible, additional experiments to address these concerns.

3 comments

We thank the reviewer for the detailed comments and address the stated weaknesses below. All changes are in the uploaded revision of our paper.

1.a Heterogeneity across viral families

We agree that filtering does not degrade performance uniformly across viral families, and that the effect is concentrated in the human-infecting shell and in Retroviridae in particular.

We see this heterogeneity as one of the study's findings rather than as a limitation. We included Figure 4 and Table 5 to test whether filtering degrades performance consistently across families, and they show that it is not consistent.

On whether our conclusions rest on this one family, we would direct the reviewer to Appendix B.7, in which we perform a per-family recovery analysis as part of this manuscript revision.("Assay-level recovery after viral fine-tuning"). Here, we assess recovery separately for each of the 11 EVEREST families (Figure 34). We count a viral family as recovered when its mean correlation across assays reaches that of the unfiltered model. We see that at least nine of eleven recover by step 5,000 at each size and at each tier. Allowing a tolerance of 0.05 Spearman correlation, each model and tier has at most two of 45 assays still below its unfiltered reference. Hence, the durability finding holds family-by-family and is not an artefact of the overrepresentation of Retroviridae sequences.

We also address this with a control experiment focussed on Retroviridae: Figure 25 shows that fine-tuning on a corpus with Retroviridae removed still recovers performance against the unfiltered model on viral proteins excluding Retroviridae. Recovery is therefore not simply a Retroviridae-specific effect, even though that family is over-represented in UniRef and accounts for most of the degradation from data filtering.

1.b Link between degradation and biosecurity impact

We agree this link is not firmly established, and we have added an explicit statement to the Limitations to that effect: our metrics measure protein-modelling capability on viral proteins, not the capacity to cause harm, and, to the best of our knowledge, no benchmark establishing that mapping yet exists for this class of models. We also want to note that such a benchmark would have large misuse potential as it could allow the creation of models with greater capacity to cause harm, and so should be approached with caution.

However, our evaluations do follow current practice for this class of model, as set out in recent best-practice guidance on biological risk evaluations for open-weight bio-foundation models [1]. ESM3 [2] and Evo 2 [3] both use viral sequence modelling and zero-shot mutation effect prediction to assess biosecurity-motivated data filtering. Among our evaluations, mutation effect prediction on human pathogen proteins is the most directly interpretable since one misuse pathway for this class of model is ranking variants of a pathogen protein by predicted fitness, which is precisely what mutation effect prediction on viral assays measures.

1.c How does this differ from machine unlearning?

Unlearning removes the influence of specific data from an already-trained model, and pre-training filtering prevents acquisition by excluding the data beforehand. We study the latter but do not argue that it should be preferred.

As a baseline, the current evidence for LLMs indicates filtering is the more tamper-resistant of the two approaches, requiring orders of magnitude more fine-tuning compute to undo than post-training interventions including unlearning [4, 5]. Our result is that even this stronger safeguard is removable for under 80 USD of H100 time, and so even if unlearning enabled cheaper recovery still, the absolute change in cost would be negligible for an attacker.

Our responses to weaknesses 1.2 and 1.3 follow in a second comment.

References

[1] B. Wei et al., "Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models," arXiv preprint arXiv:2510.27629, 2025.

[2] T. Hayes et al., "Simulating 500 million years of evolution with a language model," Science, vol. 387, no. 6736, pp. 850–858, 2025. doi:10.1126/science.ads0018.

[3] G. Brixi et al., "Genome modelling and design across all domains of life with Evo 2," Nature, vol. 652, pp. 1349–1361, 2026. doi:10.1038/s41586-026-10176-5.

[4] K. O'Brien et al., "Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs," International Conference on Learning Representations (ICLR), 2026. arXiv:2508.06601.

[5] N. Rathi and A. Radford, "Shaping capabilities with token-level data filtering," arXiv preprint arXiv:2601.21571, 2026.

2. Persistence under downstream supervised fine-tuning

We agree this is an important question, and have run the three fine-tuning settings. The revision adds Appendix B.8 ("Supervised fine-tuning of the filtered, unfiltered and recovered 3B models"), comparing 3B checkpoints under LoRA supervised fine-tuning on ProteinGym. Here, we split assays by whole protein group so that held-out assays test generalisation to unseen proteins. All three arms receive the same 3,000 fine-tuning steps, and every contrast below is against the unfiltered model fine-tuned concurrently under the same regime, measured on 18 held-out viral assays spanning 14 protein groups.

Before fine-tuning, the unfiltered model outperforms the filtered one by 0.116 Spearman on viral assays. Fine-tuning on non-viral assays only narrows that gap to 0.091, and fine-tuning on a mixture of viral and non-viral assays narrows it to 0.063, closing 46%. Both gaps are against the unfiltered model given the same fine-tuning. Holding pre-training fixed, switching the fine-tuning dataset from non-viral to mixed improves performance by an additional 0.022 Spearman (95% bootstrap CI 0.006–0.038). On non-viral assays, no contrast exceeds 0.011 Spearman and none approaches significance.

Filtering effects, therefore, do persist through downstream supervision as neither supervised fine-tuning regime overcomes even half the performance gap attributable to pre-training. By contrast, a filtered model given 5,000 steps of continued pre-training on the withheld corpus beforehand ends within 0.002 Spearman of the unfiltered model after the same fine-tuning. This presents an order of magnitude smaller residual than either fine-tuning-only arm, though of course also uses more compute.

3. Training budget and convergence

On the equal token budget: this follows ESM-2 itself, which uses the same budget across model sizes. We think this is also the more realistic comparison, reflecting how these model families are trained.

On training for fewer steps than ESM-2: Figure 9 compares our unfiltered models with the corresponding ESM-2 checkpoints, showing that mutation effect prediction is close across all five sizes, within 0.016 Spearman, but our cross-entropy is higher at 650M (+0.076 nats) and 3B (+0.125 nats), with the gap widening as models scale. Our larger models are therefore further from convergence, which we acknowledge in the Limitations.

The revision adds Appendix B.5 ("Filtering effects over the course of pretraining"): at 650M the increase in viral validation loss from filtering grows at every snapshot through 150,000 steps, while the non-viral increase stays below 0.006 nats (Figure 28), and the same trend holds at 3B (Figure 29). Longer training would therefore likely widen the filtering effects we report.

Unfortunately, training to ESM-2's 500,000 steps across five sizes and five filtering conditions is not feasible with our compute budget. For concreteness, our current training runs for the three 3B models alone cost over 100,000 USD, meaning that full training runs would cost at least an additional 300,000 USD.

We hope these responses address the reviewer's concerns. If concerns remain, or if there are further changes that would strengthen the paper, we would be grateful to hear them during the discussion period. If the reviewer considers the concerns resolved, we would be grateful if they would consider revising their score.

Dear Reviewer Jb2V,

We hope our responses and additional results have sufficiently addressed the reviewers' concerns. Thank you for your time and consideration.

Thank you,

Authors

Reviewer_nY3WOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines the effectiveness and durability of pretraining data filtering in protein language models. The authors pretrain ESM-2 models ranging from 8M to 3B parameters after removing viral proteins and their homologues from the pretraining corpus, then evaluate the resulting models on masked language modelling, mutation effect prediction, and contact prediction. Filtering degrades performance on viral proteins while leaving non-viral performance largely intact. The authors further show that fine-tuning on the excluded viral sequences recovers the lost performance at relatively low compute cost.

Critical strengths

The paper addresses an important question: whether pretraining data filtering is an effective safeguard for biological foundation models. The experiments are extensive, spanning several model sizes, filtering strategies, and evaluation tasks. I found the recovery analysis particularly valuable, as it provides concrete evidence on the limits of filtering as a durable intervention.

Critical weaknesses
  1. The link between the evaluation tasks and biological safety is not established. The paper is motivated by the prevention of biological misuse, but the experiments measure masked language modelling, mutation effect prediction, and contact prediction. It remains unclear how a given drop on these metrics translates into reduced capability for misuse, which limits what the results can tell us about safety in practice.

  2. The authors treat performance as recovered once the average score matches that of the unfiltered model on the same metric.However, matching averages do not imply that capability has been restored uniformly across individual proteins or viral families. This matters because Figures 4 and 5 indicate that most of the filtering effect is driven by a small number of families.

  3. The recovery cost presumes access to the excluded viral sequences. The reported figure of under USD 80 covers only the fine-tuning compute and omits the effort required to locate, curate, and prepare suitable data. It is therefore unclear how closely this number reflects the practical cost of reversing the safeguard.

  4. The effect of finetuning on non-viral performance is underexplored. Figure 18 shows that non-viral language modelling performance degrades during fine-tuning for the larger models, suggesting that recovering viral capability may come at the expense of other capabilities. A more explicit discussion of this trade-off would strengthen the paper.

Questions for authors
  1. To what extent do these protein modelling tasks measure the biological capabilities that filtering is intended to reduce?

  2. Given that most of the filtering effect stems from a small number of viral families, is recovery distributed across families, or is it similarly concentrated?

  3. How much could random initialisation and training variability influence the reported scaling trends, given a single run per configuration?

  4. At the checkpoint where viral performance is deemed recovered, how much non-viral performance has been lost?

  5. How would the conclusions about recovery cost change if access to the excluded viral training data were restricted?

2 comments

We thank the reviewer for these insightful questions and address the reviewer's stated weaknesses and questions below. All changes described are in the uploaded revision.

Link between the evaluation tasks and biological risk

We agree that our metrics measure protein-modelling capability on viral proteins rather than the capacity to cause harm. We believe that no benchmark establishing that mapping exists for this class of model. Note that such a benchmark would itself pose substantial dual-use risks. Nonetheless, we have added a statement to the Limitations acknowledging this. Please also see the response to the other reviewer.

Averages may hide non-uniform recovery

We agree there is an important alternative notion of recovery, tracking performance on individual proteins. To measure this form of recovery, we compute the fraction of individual assays that reach or exceed their own unfiltered reference at fine-tuning step 5,000. At strict parity this is 34 of 45 assays (76%) at 3B tier-0 and 36 of 45 (80%) at tier-1, rising from 89% at 8M. Hence we see that the fraction falls as the models scale in parameter count, consistent with our finding that filtering becomes more effective with scale. Allowing a tolerance of 0.05 Spearman, every evaluated model and tier falls short on at most two of 45 assays. The largest residual shortfall anywhere is 0.136 Spearman, against a filtering effect being recovered that reaches 0.389 Spearman for Retroviridae at 3B. In our revised manuscript, we plot the percentage of assays, and of families for which every assay passes, against the allowed shortfall ε from 0 to 0.10 (Figure 32), so the reviewer can read off the fraction recovered at whichever tolerance they consider meaningful.

Recovery is concentrated in the same families as the filtering effect. At 3B tier-0 the two families where nearly all the performance degradation occurs, Retroviridae (mean deficit 0.389) and Picornaviridae (0.240), both return to parity or better after fine-tuning (−0.009 and −0.057). Recovery occurs in most other families too: the mean Spearman on at least 9 of 11 families recovers at every model size and tier.

Recovery cost presumes access to the excluded data

We agree that our cost estimates only count fine-tuning compute and exclude the cost of locating, curating and preparing suitable sequences. We have added this caveat to the Limitations. We would note, though, that for the models we study the excluded sequences are public UniRef records, so for any model pre-trained on UniRef the curation cost is unlikely to prohibit an actor from circumventing the safeguard.

Non-viral performance lost at the recovery checkpoint

At 650M, where we score the non-viral control before fine-tuning, non-viral cross-entropy rises by 0.017–0.026 nats (tier-0) and 0.011–0.016 nats (tier-1) by the checkpoint at which viral performance has recovered, depending on the shell we measure recovery on. Therefore, there is a measurable cost to recovering performance on viral proteins via fine-tuning. However, in comparison to the performance gap recovered on viral proteins (0.36–0.44 nats on the most affected shells), this cost is relatively small. The revision now reports this trade-off in the main text.

Seed variability

The use of a single seed is indeed a limitation. Pretraining all our 3B models alone cost over 100,000 USD worth of GPU hours, and so training large models across multiple seeds was not feasible . However, we want to note that for each model size the filtered and unfiltered models are initialised identically and differ only in their pretraining data, so the comparison is not between independently initialised runs.

We hope these responses resolve the reviewer's concerns. If further concerns remain, or there are changes that would strengthen the work, we would be grateful to hear them during the discussion period. If the reviewer considers their concerns addressed, we would be grateful if they would consider revising their score.

Dear Reviewer nY3W,

We hope our responses and additional results have sufficiently addressed the reviewers' concerns. Thank you for your time and consideration.

Thank you,

Authors

Reviewer_NYQbOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The work examines whether filtering of viral data before pretraining is an effective means for safeguarding against dual use bio capabilities. Unfortunately, it finds that it does not provide a robust safeguard for open source models.

Critical strengths

This work seems to have been carefully done and obviously required effort as it involved pretraining several models (though relatively small), overall it seems like a valuable contribution to get one more datapoint on what's an effective or ineffective safeguard (with more caveats on limitations below).

Critical weaknesses
  • The main concern I have is that the work does not do enough to engage with further limitations beyond what's already stated. It is well known that adaptive attacks can bypass seemingly effective safeguards (see, e.g., https://arxiv.org/abs/2412.07097, https://arxiv.org/abs/2002.08347, https://arxiv.org/abs/2409.18025). The paper, however, only tests narrow sets of adaptive attacks in the fine-tuning setting (not the inference-time setting). It is likely worth acknowledging that there may exist an adaptive attack that changes the results here. Importantly, it's worth acknowledging that MSM and the other tested metrics may not be sufficient for assessing dual use risk and are still proxy measures for downstream risk of harm.
  • It's less clear how much context window size matters here. 1024 is quite small and I do wonder if results will continue to hold with larger context window sizes where few-shot prompting might become more feasible. Some discussion of this would be helpful (e.g., how does context window relate to fitting in a viral sequence or the important parts of one).
Questions for authors

How does context window size relate to the viral sequences at play? How do you expect your findings to change with larger context window sizes?

3 comments

We thank the reviewer for their time, and for recognising the value of this work. We address both weaknesses and the question below and now acknowledge the two limitations the review raised explicitly in the revision.

Untested adaptive attacks

The reviewer is right that we did not test adaptive attacks beyond fine-tuning, and we agree other attacks may recover the lost performance more cheaply. We would note that this strengthens rather than weakens our conclusion. Fine-tuning on sequences from the removed families already restores almost all of the filtered performance: 34 of 45 EVEREST assays reach strict parity at 3B tier-0, and at most two of 45 fall short of the unfiltered reference model by more than 0.05 Spearman (Appendix B.7). In the most expensive case we measure, masked language modelling at 3B, this takes 14–24 exaFLOPs, or 47–79 USD of H100 time at 34% model FLOPs utilisation. A stronger attack might lower this cost even more, but would not change our conclusion that this safeguard is brittle.

We have added a paragraph to the Limitations section naming the attacks we did not test (direct weight edits, in-context learning, retrieval augmentation), citing the work above.

Metrics are proxies for dual-use risk

We fully agree, and we believe this is an open problem for the field. Our metrics follow current practice for this class of models, as surveyed in recent best-practice guidance on biorisk evaluations for open-weight bio-foundation models (Wei et al., 2025).

We have added a statement to the Limitations section: our evaluations measure protein-modelling capability on viral proteins, not the capacity to cause harm. Benchmarks establishing a mapping from changes in cross-entropy or Spearman correlation to changes in dual-use risk do not yet exist for this class of model.

We would like to note that benchmarks that would allow a stronger inference to changes in dual-use risk would themselves pose substantial misuse potential and so should be developed with caution. This would require transparent governance, legal advice and compliance with the biological weapons convention.

Context window size

While 1024 tokens is a small context window for modern LLMs, it accommodates almost all proteins in our dataset: roughly 98% of viral and 97% of non-viral proteins in UniRef50 are at most 1,022 residues, which is the usable limit once the <cls> and <eos> tokens are counted.

This is also the context length of the original ESM-2 models, whose architecture and training recipe we follow. Our evaluations aim to compare models of the ESM-2 architecture, all of which share this context window length. It therefore cannot account for any of the filtering effects we observe.

On few-shot prompting: we agree that a model with a substantially larger context window, an expanded vocabulary admitting in-context labels, and training that rewarded use of in-context information could plausibly restore much of the lost performance through in-context learning, as O'Brien et al. [1] suggest for large language models. Our Limitations notes this for retrieval-augmented protein language models such as Protriever [2] and Tranception [3]. Testing this would require changing the architecture and training procedure of our models, which puts it beyond this paper's scope. As above, a positive result would only strengthen our conclusion that data filtering is a safeguard that is straightforward to circumvent.

We hope these responses address the reviewer's concerns. If any remain, or if there are further changes that would strengthen the paper, we would be glad to hear them during the discussion period. If the reviewer considers their concerns resolved, we would be grateful if they would consider revising their score.

References

[1] K. O'Brien et al., "Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs," International Conference on Learning Representations (ICLR), 2026. arXiv:2508.06601.

[2] R. Weitzman et al., "Protriever: End-to-End Differentiable Protein Homology Search for Fitness Prediction," International Conference on Machine Learning (ICML), 2025.

[3] P. Notin et al., "Tranception: Protein Fitness Prediction with Autoregressive Transformers and Inference-Time Retrieval," International Conference on Machine Learning (ICML), 2022.

Thank you for the response. I think in general, this is an interesting paper, and am for including negative results. But I still think this particular negative result has somewhat of a limited generality due to the limitations discussed above and in the paper. It would probably be too big a lift (a whole other paper) to provide a more general negative result that can more accurately inform policymaking and safety methods in this area. As such, I'm going to leave my rating as is (leaning acceptance but only weakly so).

Thank you for taking the time to read our response, as well as for your constructive feedback throughout the review process. We genuinely appreciate your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.