acceptodds
Under review as a conference paper at ICLR 2027

DETER: Suppression During Pretraining for Tamper Resistance

Abstract

Open-weight language models can acquire knowledge that could enable biological, chemical, or cyber misuse. Any attempt to limit such knowledge must remain effective after the model weights are released and can be fine-tuned. Unlearning applied after pretraining can be rapidly undone, whereas filtering the pretraining corpus offers greater resistance. We ask whether actively suppressing knowledge during pretraining can further increase the fine-tuning required to acquire it. Even simple gradient ascent interleaved with standard pretraining makes medical knowledge harder to acquire through adversarial fine-tuning than filtering alone or post-hoc unlearning. At 0.5B parameters, reaching the held-out PubMed loss of an unfiltered model requires over 13 as many PubMed fine-tuning tokens as after filtering alone. Retained knowledge is evaluated using both held-out data and downstream tasks. Across model sizes, varying suppression strength reveals an increasingly favorable trade-off between fine-tuning difficulty and held-out loss on retained pretraining data. These results point to pretraining as an effective stage for making unwanted capabilities harder to acquire through subsequent fine-tuning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
tamper resistance, unlearning, pretraining, adversarial fine-tuning, open-weight models, data filtering

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ML84OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents DETER, which combines pretraining data filtering with periodic gradient-ascent unlearning during pretraining stage on selected target corpus to increase the cost of subsequent relearning, achieving a tamper resistance. The paper implements series of experiments which target medical knowledge in language models ranging from 113M to 521M parameters and compares against document filtering, token-level filtering, and post-hoc unlearning. Recovery is measured by the PubMed finetuning tokens needed to reach the unfiltered baselines' PubMed loss. At 512M setting, after unlearned by DETER, it requires approximately 13.5 times more tokens than document filtering alone, with a relatively small increase in general-domain loss.

Critical strengths
  1. The paper uses a periodic gradient ascent on target-domain data in ordinary pretraining, achieving effective suppression of medical knowledge and increasing the difficulty of recovering it through subsequent finetuning. DETER adds only a few training compute while substantially increasing the finetuning data required for recovery—by up to 13.5× over filtering alone under the paper’s PubMed loss criterion. Besides, DETER can potentially become a interpretable method for other domain in LLM.

  2. The paper implement a large number of experiments including comparison with baselines and ablation studies.

Critical weaknesses
  1. Justification of the recovery criterion: In this paper, the unfiltered baseline's PubMed corpus loss is used as the criterion to judge if the unlearned model has been successfully recovered, however, in llm the improvements in perplexity do not necessarily translate into better downstream task performance.
  2. Understanding the source of increased recovery difficulty: DETER requires more finetuning tokens to reach the reference PubMed loss, but it remains unclear whether this primarily reflects its higher loss after unlearning or whether the parameter configurations induced by suppression make subsequent optimization more difficult.
  3. There are more baseline methods needed to be evaluated. And more seeds for pretraining need to be evaluated
  4. Ambiguity in the scope of the claims: Since all target domain suppression and recovery experiments focus on medical knowledge, the main claims should be explicitly scoped to this setting. Although the limitations section acknowledges this boundary, reflecting it consistently in the abstract and contribution statements would make the contribution clearer.
Questions for authors
  1. Could authors clarify why corpus loss is an appropriate criterion for capability recovery, and whether it offers advantages over accuracy or task success evaluations in this medical domain setting? The evidence that can establish the correspondence between this criterion and actual task performance would strength the evaluation. Otherwise, supplementing current results with direct performance metrics would provide more convincing evidence of whether recovery has occurred.
  2. DETER requires more finetuning tokens to reach the reference PubMed loss. It would be helpful if the authors briefly discussed how this result may reflect both the higher loss after unlearning and possible changes in subsequent optimization dynamics arising from the learned parameter configurations. This is a request for clarification rather than additional experiments, and does not dispute the method’s observed unlearning effects. A clearer account of the underlying mechanism would strengthen the contribution.
  3. Given the relevance of SGTM and TAR to pretraining time capability removal and tamper resistance, could the authors clarify why these methods were left for future comparison rather than included in the current evaluation?
  4. The experiments use a single pretraining seed per configuration, while the reported repetitions vary the fine-tuning seeds. Could the authors discuss how pretraining variability might affect the reported gains, particularly the scaling trend?
  5. It will make the paper clearer if authors can make the claim clearer.

If authors can provide reasonable explanations for 1. 2. 3. 4. I can keep the decision as weak acceptance, otherwise I will consider dropping the score to the weak rejection

Reviewer_WqJEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DETER, which interleaves gradient ascent on target-domain documents with ordinary pretraining, so that this knowledge is harder to recover through later fine-tuning. Using medicine as a proxy domain, the authors pretrain 113M, 224M and 521M models from scratch on FineWeb-Edu with the most medical 20% of tokens filtered out, and take one suppression step on the most medical 5% every 50 pretraining steps, with separate optimizer states for the two objectives.

Resistance is measured as the number of PubMed fine-tuning tokens needed to reach the PubMed loss of an unfiltered model. DETER needs 3.1, 4.1 and 13.5 times as many tokens as filtering at the three sizes, compared with about 1.3 to 1.5 times for token filtering and post-hoc RMU, with a small increase in general loss. Ablations cover the suppression objective, separate versus shared optimizer states, suppression onset, suppression without filtering, and token-level suppression.

Critical strengths
  1. Simple idea with a clean controlled study. Models are pretrained from scratch at three sizes and compared against filtering, token filtering and RMU with matched budgets, using both corpus and task losses and three fine-tuning seeds. The gain over filtering grows with scale, which is the most interesting result.

  2. Informative ablations. The optimizer-state ablation is a useful warning: shared states make suppression look stronger before fine-tuning, but the extra effect disappears within a few fine-tuning steps. The comparisons of suppression objectives and onsets also give practical guidance.

Critical weaknesses
  1. The resistance is small in absolute terms and tested against a limited attack. Every method reaches the unfiltered PubMed loss within the attack budget, and at 521M DETER needs only 1.58M tokens, about 0.015% of pretraining. The only attack is full fine-tuning on PubMed with a tuned learning rate. Each configuration uses a single pretraining seed, so the jump from about 4 times to 13.5 times between 224M and 521M, which drives the scaling claim, rests on one run per size.

Questions: How stable is the 521M result across pretraining seeds? How does DETER hold up under other attacks, such as fine-tuning on medical QA data, LoRA, or mixing in retain data? Since gradient ascent can raise loss in a shallow way, do accuracy or few-shot results after fine-tuning show the same gap, and is there evidence that the knowledge is absent rather than masked?

  1. Support for the broader safety motivation is limited. Models are at most 521M, capability is measured by loss rather than accuracy, the same classifier defines both the suppression data and the in-domain evaluation splits, and medicine is only a proxy for hazardous domains. Tamper-resistance baselines such as SGTM and TAR are not compared, and the extra compute of suppression is not accounted for.

Questions: Can you add at least one of these baselines, and test a second target domain?

Minor issues: The Ethics Statement section is empty.

Questions for authors

Please see the weaknesses above.

Reviewer_hD6GOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a language model can be made hard to fine-tune toward a target domain by suppressing that domain during pretraining. The experiments use medical knowledge as a stand-in for hazardous knowledge. The method, DETER, removes medical documents from the pretraining corpus as data filtering does, and at regular intervals also takes gradient ascent steps on some of the removed documents, with optimizer statistics kept separate from ordinary pretraining. Resistance is the amount of PubMed fine-tuning an adversary needs to bring the model's PubMed loss back to that of a model trained without filtering, relative to filtering alone. The authors report that suppression raises this cost above filtering, token-level filtering and post-hoc unlearning, with limited damage to retained capabilities, and that the advantage grows with model size up to the largest model studied. They argue that pretraining is an effective stage for making unwanted capabilities harder to acquire.

Critical strengths

1. Plain gradient ascent becomes effective when interleaved with pretraining. Interleaved with ordinary pretraining steps, the simplest suppression objective makes PubMed fine-tuning costlier than filtering, token filtering and post-hoc RMU at every model size, with the largest gap at the largest size (Table 1, Fig. 4), and it gives the best trade-off among the suppression objectives compared (Fig. 6). This weighs most in my assessment because gradient ascent is usually seen as a blunt post-hoc tool, and the paper shows a simple way to make it useful.

2. The attacker is tuned with care. The adversary's learning rate and schedule are chosen for each model to reach the target loss as quickly as possible, and at the largest size the attack is extended well past the parity point while medical task loss is tracked alongside PubMed loss (Fig. 3). This matters because it makes the relative comparisons credible within the attack family the paper tests.

Critical weaknesses

1. The paper does not isolate pretraining as the stage that matters.

  • The latest onset gives the highest mean resistance at every model size (Table 2), and the authors report that using the shorter interval of a late start from the beginning does not reproduce this (p.6). This points toward the end of training rather than toward pretraining as such.
  • The post-hoc baseline, RMU, differs from suppression in its objective as well as its timing (p.3). No run continues training from the finished filtered model with the same interleaved gradient ascent schedule, so the gain could come from interleaving rather than from pretraining.
  • The conclusion says that at the largest size "earlier suppression improves the balance between resistance and general loss" (p.9). In Table 2 the two onsets have the same general loss, and the earlier one is less resistant. Only the task losses favour it.

2. Collateral damage is larger than "limited" suggests.

  • Before any attack, the suppressed model at the largest size has the worst general task loss of all methods (Table 1). The caption's "little change in general loss" refers only to corpus loss.
  • Suppression raises biology corpus loss far more at the largest size than at the smaller ones, to the highest value of any method (Fig. 2). Table 1 reports biology only through MMLU-biology task loss, where suppression at the largest size looks no worse than token filtering and better than RMU.
Questions for authors
  • How much resistance remains if the adversary first retrains the output layer?
  • Do probes on hidden layer's activations still find medical knowledge in the suppressed models?

Comments on the paper

1 comment

Hi Reviewers,

The authors have provided responses. Please check if they have addressed your concerns.

Thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.