acceptodds
Under review as a conference paper at ICLR 2027

Preemptive LLM Unlearning against Forbidden Capability Acquisition via Gradient Sealing

Abstract

Open-weight LLMs are released not only as fixed products but also as substrates for downstream fine-tuning. This openness, however, creates legal and ethical risks because users may misuse fine-tuning to instill illicit knowledge or enable hostile operations. Model providers therefore need a pre-release defense against such acquisition, motivating the problem of *preemptive unlearning*. Unlike *retrospective unlearning*, which removes capabilities already present in a fixed model, preemptive unlearning seeks to prevent their acquisition under unseen attack data and future fine-tuning procedures. Despite its practical importance, this setting remains largely unexplored, presents distinct challenges, and is therefore the central focus of our work. We first verify that existing retrospective methods provide insufficient pre-release protection. Even when forbidden capabilities are suppressed in current outputs, forbidden-domain data can still induce gradients through internal pathways, enabling later acquisition. Motivated by this finding, we propose a *gradient-sealing principle* that blocks these pathways by pushing relevant pre-activations into the negative region, where ReLU-family activations exhibit zero or near-zero derivatives. Experiments across multiple LLM families demonstrate our stronger resistance to downstream acquisition than retrospective baselines, validating gradient sealing as an effective mechanism for pre-release protection.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM unlearning, large language models, machine unlearning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Nx7qOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Gradient Sealing Unlearning (GSU), a preemptive defense against the reacquisition of forbidden capabilities through downstream fine-tuning. The method consists of three stages: exposing potential acquisition pathways through proxy fine-tuning, localizing sensitive activation channels, and sealing these channels by pushing their pre-activations toward the negative saturation region of the SiLU activation function. The idea is interesting, and the experimental evaluation covers multiple model architectures, benchmarks, and existing defense methods.

Critical strengths
  1. The paper distinguishes a model’s current capabilities from its ability to acquire them through fine-tuning, offering a valuable perspective on LLM unlearning.
  2. The paper provides several complementary analyses supporting the proposed gradient-sealing mechanism.
Critical weaknesses
  1. The method uses a soft penalty to push selected pre-activations toward a region with small derivatives. However, “sealing” could be mistaken for freezing weights or removing neurons. Please clarify that it changes the responses of selected gates without imposing hard constraints on parameter updates.
  2. The evaluation clearly specifies that released models must retain at least 95% of the reference utility, while usable attack checkpoints must retain at least 90%. A brief explanation of why a slightly lower threshold is allowed after fine-tuning would make the protocol easier to interpret. This is a request to clarify the existing protocol choice.
Questions for authors
  1. Could the authors clarify how gradient sealing differs from freezing or removing neurons? Specifically, does sealing use a soft loss penalty to modify gate responses while keeping all model parameters trainable?
  2. What motivates the utility thresholds of 95% for released models and 90% for usable attack checkpoints? Is the lower attack-time threshold intended to accommodate some utility loss during fine-tuning while excluding attacks that severely degrade model utility?
2 comments
AuthorsOpenReview ↗

Response to Reviewer Nx7q

Thank you for your positive review, clear acceptance recommendation, and recognition of our problem framing, simple design, and broad evaluation. We address both clarification points below and will include them in the revision.


Weakness 1: What “sealing” changes: soft gate-response control, not freezing or removal

“The method uses a soft penalty to push selected pre-activations toward a region with small derivatives. However, “sealing” could be mistaken for freezing weights or removing neurons. Please clarify that it changes the responses of selected gates without imposing hard constraints on parameter updates.”

We will clarify in §4.3 that sealing is a soft, training-time penalty on gate responses—not freezing, masking, or pruning:

  1. Soft hinge penalty during defense only: Lseal\mathcal{L}_ \text{seal} (Eq. 4) applies max⁡(uℓji(θ)−τ,0)2\max(u_ {\ell j}^i(\theta)-\tau,0)^2 to the sparse localized gate set G\mathcal{G} (1%–10% of channels in K∈{1,2,4}K\in\{1,2,4\} layers), nudging forbidden-input pre-activations into the SiLU negative tail (u≤τ<0u\le\tau<0) with zero direct gradient once u≤τu\le\tau and no hard clamp or weight mask.

  2. All weights stay 100% trainable during attack: Full-parameter AdamW updates every parameter freely with no sealing loss or frozen units. Across 30 epochs (390 updates) on Qwen3.5-2B:

    Table C.1: Gate sensitivity and parameter updates during 30-epoch attack.

    QuantityGSUGSU w/o seal (2× suppression)
    Mean abs. SiLU'(u) at epoch 300.3915460.442273
    Gate / Up / Down update RMS (×10⁻⁸)6.077 / 7.587 / 8.2495.617 / 6.589 / 7.223

    Selected-channel weights actively move at every step (slightly larger RMS than control), while their mean |SiLU'(u)| remains 11.47% lower (0.391546 vs. 0.442273).

  3. No neurons removed; utility stays intact: GSU preserves 98.05% of reference utility after 30 attack epochs (100.18% at release in Table A.1) and maintains benign fine-tunability (Fig. 4c).


Weakness 2: Why released models use 95% utility and attack checkpoints 90%

“The evaluation clearly specifies that released models must retain at least 95% of the reference utility, while usable attack checkpoints must retain at least 90%. A brief explanation of why a slightly lower threshold is allowed after fine-tuning would make the protocol easier to interpret. This is a request to clarify the existing protocol choice.”

The two thresholds govern different roles in the attack–defense game (Defs. 2.1–2.2):

  1. ≥95% at release constrains the defender (ϵr\epsilon_r, Def. 2.2): Every defended release must preserve at least 95% of UrefU_\text{ref} via UWC before any attack.
  2. ≥90% during attack filters the attacker (ϵu\epsilon_u, Def. 2.1, defining UA90): Narrow-domain fine-tuning without retain data naturally causes minor utility drift (2%–5%). Setting the attack bar at 90% allows normal drift while disqualifying degenerate attacks (e.g., extreme LRs) that memorize targets only by collapsing general utility.
  3. Results sit well above both cutoffs: In our 30-epoch Qwen3.5-2B runs, GSU and NPO+SAM retain >98% utility (98.05% for GSU), so UA90 never hinges on the 90% vs. 95% gap. We will add this to §5.1 and Appx. E.3.

Question 1: How does gradient sealing differ from freezing or removing neurons?

“Could the authors clarify how gradient sealing differs from freezing or removing neurons? Specifically, does sealing use a soft loss penalty to modify gate responses while keeping all model parameters trainable?”

Yes (see W1, Table C.1): sealing uses a soft hinge penalty only during defense training, becoming inactive once u≤τu\le\tau with no hard constraints or pruning. During downstream attacks, full-parameter AdamW updates every weight normally (RMS 6.077–8.249×10⁻⁸), while targeted gates retain an 11.47% lower mean |SiLU'(u)| (0.391546 vs. 0.442273) and 98.05% reference utility.


Question 2: What motivates the 95% and 90% utility thresholds?

“What motivates the utility thresholds of 95% for released models and 90% for usable attack checkpoints? Is the lower attack-time threshold intended to accommodate some utility loss during fine-tuning while excluding attacks that severely degrade model utility?”

Exactly (see W2): 95% enforces high utility on the defender's release, while 90% filters attacker checkpoints in UA90—accommodating minor fine-tuning drift while excluding destructive attacks. Since our 30-epoch checkpoints retain >98% utility, all reported UA90 results sit well above both bars.

Area_Chair_HxYfOpenReview ↗

Please engage in the discussion with authors

Dear Reviewer Nx7q,

Please engage in the discussion with authors, thank you!

AC

Reviewer_yHEnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Gradient Sealed Unlearning, a method (which the authors categorize as "preemptive unlearning") that aims to prevent open-weight language models from acquiring forbidden capabilities through future fine-tuning. Unlike traditional unlearning methods, which suppress knowledge already present in a model, the authors argue that preemptive unlearning should also constrain the gradient pathways that future fine-tuning may exploit to learn forbidden knowledge. The proposed method consists of three stages. At first a copy of the original model is finetuned on a proxy forbidden dataset to identify changes in internal activations. Then it selects layers and MLP activation gates that have the largest positive pre activation shifts. Finally, the original model is optimized using a new loss that pushes the selected pre-activations into the negative tail of the SiLU activation function, where local derivatives become small. The loss has other terms for suppression and retention to preserve the models normal utility.

Critical strengths

S1. The distinction between retrospective and preemptive unlearning is interesting and shows an important safety threat.

S2. The proposed algorithm is relatively simple and intuitive. The idea of identifying learning-sensitive activation gates through a temporary fine-tuning process, and subsequently controlling their gradients through pre-activation regularization, is an interesting direction.

S3. The experiments cover different model sizes and families. The use of disjoint defender and attacker datasets is also a useful setup for evaluating generalization beyond the proxy training examples.

Critical weaknesses

W1. My main concern is that the paper (and the results) do not sufficiently distinguish suppressing forbidden capability at release and making subsequent learning of that capability more difficult. The main results in Tables 1 and 2 show lower forbidden domain performance after a fixed number of fine-tuning updates. However, a model starting from a lower initial capability may also have lower performance after fine-tuning, even if its ability to learn is not meaningfully reduced. As a result, these results alone do not prove that gradient sealing makes future capability acquisition more difficult.

W2. In fact, the appendix results in F.4 raise some concerns about my previous point. On Qwen3.5-2B, the method shows a larger increase in extraction strength during the attack for NPO+SAM. Although gsu starts from a lower extraction score, the relative increase is considerably larger. This seems difficult to reconcile with a strong claim of reduced learning susceptibility.

W3. The ablation in Appendix F.2 shows that the difference between gsu and the variant without the sealing objective decreases substantially with additional attack iterations. This shows that the method is highly dependent on the assumption of the attacker having very small budgets. I appreciate the authors acknowledge this limitation, but I believe it is an important limitation.

W3. The theoretical analysis in provides a local upper bound for one-step capability acquisition in terms of the projected attack gradient. However, it does not directly imply that the proposed sealing objective reduces this projected gradient, or that the reduction will persists throughout fine-tuning. Similarly, the empirical gate analysis measures local activation derivatives at release rather than the relevant parameter gradients during acquisition. This leavest a gap between the connection between the proposed method and the claimed acquisition resistance. The reason I think this is important is that it is difficult to determine whether GSU introduces a genuine barrier for future learning, or primarily changes the initial model state in a way that provides better performance under a limited attack budget. Both may be useful, but they are different contributions and should be distinguished.

W4. the experimental evaluations are relatively narrow compared with the threat model considered in the paper. The authors assume an attacker with full access to the released model parameters, who can choose the finetuning objective, optimizer, schedule, and training configuration. However, the implemented evaluation mainly considers standard supervised finetuning with a fixed optimization procedure.

W5. The attack experiments use has a fixed learning rates and no warmup. It is unclear whether the proposed gradient attenuation would remain effective under other optimizers, learning-rate schedules, or hyperparameters optimized specifically against GSU.

W6. The experiments evaluates disjoint datasets but the separation is still limited. These setup does not show whether the selected gradient pathways generalize to very different datasets with different distributions of forbidden knowledge. I think a setup where an attacker tries to fine-tune and acquire a completely different unsafe capabililty is more realistic.

W7. I think the authors have unnecessarily limited themselves to unlearning literature. There is a broader literature on making open-weight models resistant to malicious fine-tuning. For example, these are some related works that are missing from the paper's evaluations: Tamper-Resistant Safeguards for Open-Weight LLMs (ICLR 2025), Self-Degraded Defense against Malicious Fine-tuning (ACL 2025), Defending Harmful Fine-tuning via Geometric Bottlenecks (ICML 2026), A Robust Unlearning Method with Adaptive Knowledge Guidance and Memory Preservation (AAAI 2026). They address a closely related problems.

Questions for authors

Q1. Can the authors comment on how much of the improvement comes from reducing future learning, rather than simply suppressing the initial forbidden capability? In particular, Appendix F.4 shows that GSU experiences a larger increase in extraction strength than NPO+SAM over the same fine-tuning budget. How should this result be interpreted in the context of the proposed gradient-sealing mechanism?

Q2. How sensitive is the method to the attacker's optimizer and learning-rate schedule?

13 comments
AuthorsOpenReview ↗

Response to Reviewer yHEn (Part 1/4)

Thank you for recognizing the value of framing preemptive unlearning, the intuitiveness of GSU, and our disjoint setup. We address each comment with new controlled experiments below and will include all findings in the revision.


Weakness 1: Initial suppression versus resistance to future acquisition

“My main concern is that the paper (and the results) do not sufficiently distinguish suppressing forbidden capability at release and making subsequent learning of that capability more difficult. The main results in Tables 1 and 2 show lower forbidden domain performance after a fixed number of fine-tuning updates. However, a model starting from a lower initial capability may also have lower performance after fine-tuning, even if its ability to learn is not meaningfully reduced. As a result, these results alone do not prove that gradient sealing makes future capability acquisition more difficult.”

We agree that a lower endpoint from a lower release score does not by itself prove slower learning. Over a 3× longer attack (30 epochs / 390 updates) on Qwen3.5-2B/TOFU (disjoint), we (1) show that doubling unsealed suppression lowers release ES₀ yet worsens 30-epoch recovery, and (2) match a no-seal control on release ES and utility—confirming 11.4% less acquisition (0.269 pp lower UA90@30) from the same starting point.

  1. Stronger unsealed suppression rebounds faster over 30 epochs

    We compare GSU with GSU w/o seal (removing only Lseal\mathcal{L}_\text{seal}) and GSU w/o seal (2× suppression) (doubling NPO weight, chosen on defender data before attack).

    Table B.1: Release scores and 30-epoch recovery (%).

    MethodES₀ES₃ES₅ES₃₀UA90@30
    GSU3.4563.9254.7065.9385.944
    NPO+SAM4.4554.6924.9486.1406.140
    GSU w/o seal5.0345.1155.2946.3786.378
    GSU w/o seal (2× suppression)4.5265.0905.4426.6796.679

    GSU stays lowest at every stage (UA90@30 5.944% vs. 6.140% / 6.378%, retaining 98.05% utility). Doubling NPO drops ES₀ (5.034% → 4.526%) yet raises UA90@30 (6.378% → 6.679%)—harder release suppression without sealing actually rebounds faster.

  2. From a matched starting state, GSU still acquires less

    We calibrated a no-seal release purely on defender data to match GSU within ±0.05 pp ES₀ and ±0.20 pp utility, froze it, and ran the same 30-epoch attack:

    Table B.2: Release-matched 30-epoch comparison (%).

    MethodES₀ES₃ES₅ES₃₀UA90@30
    GSU3.4563.9254.7065.9385.944
    GSU w/o seal (matched release)3.4424.1284.9676.2076.213

    Starting together (ES₀ gap 0.014 pp; utility 100.18% vs. 100.31%), the no-seal control gains 2.765 pp vs. 2.482 pp for GSU (11.4% more acquisition) and ends 0.269 pp higher in UA90@30—consistent with Table A.8's paired seeds.

Controlling for the starting state confirms that sealing genuinely slows subsequent acquisition over 390 updates.


Weakness 2: Interpreting the larger increase in Appendix F.4

“In fact, the appendix results in F.4 raise some concerns about my previous point. On Qwen3.5-2B, the method shows a larger increase in extraction strength during the attack for NPO+SAM. Although gsu starts from a lower extraction score, the relative increase is considerably larger. This seems difficult to reconcile with a strong claim of reduced learning susceptibility.”

That is a fair observation. Comparing raw ΔES = ES₁₀ − ES₀ from unequal starting points mixes learning susceptibility with baseline headroom:

Table B.3: ES from saved F.4 runs (%; increase in pp).

MethodES₀ES₃ES₅ES₁₀ΔES (0–10)
GSU3.3793.8894.7135.3771.997
NPO+SAM4.7304.9655.1025.5330.803
  1. Lower at all 11 checkpoints (epochs 0–10): An attacker never extracts more from GSU than from NPO+SAM at any stage.
  2. Headroom effect: GSU starts 1.351 pp lower (3.379% vs. 4.730%), leaving more room to rise before approaching saturation.
  3. Matched start reverses ΔES: Once ES₀ is matched in Table B.2 (3.456% vs. 3.442%), the unsealed control rises 11.4% more than GSU (+2.765 vs. +2.482 pp), showing sealing reduces subsequent learning when baselines are equal.
AuthorsOpenReview ↗

Response to Reviewer yHEn (Part 2/4)

Weakness 3a: Dependence on a small attack budget

“The ablation in Appendix F.2 shows that the difference between gsu and the variant without the sealing objective decreases substantially with additional attack iterations. This shows that the method is highly dependent on the assumption of the attacker having very small budgets. I appreciate the authors acknowledge this limitation, but I believe it is an important limitation.”

We revisited F.2 and extended the default attack 3× to 30 epochs (390 updates):

Table B.4: F.2 paired ablation (%).

MethodES₀ES₅ES₁₀
GSU3.3794.7135.377
GSU w/o seal4.6105.3235.753

Table B.5: Extraction across 10–30 attack epochs (%).

EpochsGSU ES@endGSU UA90NPO+SAM ES@endNPO+SAM UA90
104.9814.9815.3125.312
205.6015.6015.9635.963
305.9385.9446.1406.140

In F.2, GSU still leads by 0.610 pp (epoch 5) and 0.376 pp (epoch 10). More importantly, across 10, 20, and 30 epochs (Table B.5), GSU stays lower than NPO+SAM in both ES@end and UA90 (capturing GSU's epoch-29 peak; both retain ≥98% utility at epoch 30), and also outperforms both unsealed ablations and the matched control at epoch 30 (Tables B.1–B.2).


Weakness 3b: From the local theoretical analysis to actual fine-tuning updates

“The theoretical analysis in provides a local upper bound for one-step capability acquisition in terms of the projected attack gradient. However, it does not directly imply that the proposed sealing objective reduces this projected gradient, or that the reduction will persists throughout fine-tuning. Similarly, the empirical gate analysis measures local activation derivatives at release rather than the relevant parameter gradients during acquisition. This leavest a gap between the connection between the proposed method and the claimed acquisition resistance. The reason I think this is important is that it is difficult to determine whether GSU introduces a genuine barrier for future learning, or primarily changes the initial model state in a way that provides better performance under a limited attack budget. Both may be useful, but they are different contributions and should be distinguished.”

To bridge our one-step bound and multi-step fine-tuning, we tracked the selected gates across 390 unrestricted AdamW updates (30 epochs), showing (1) why Proposition 3.1 motivates gate-level control, (2) that selected-gate |SiLU'(u)| stays 11.47% lower through epoch 30, and (3) that actual AdamW weight steps are slightly larger—proving the barrier comes from persistent gate-response damping rather than smaller optimizer steps.

  1. Role of Proposition 3.1: The bound ΔSF≤ηLF∥ΠSga∥2+O(η2)\Delta S_\mathcal{F}\le\eta L_\mathcal{F}\|\Pi_\mathcal{S}\mathbf{g}_ a\|_2+O(\eta^2) and Appendix C.2 show why output suppression is incomplete: identical outputs can hide different internal gate derivatives cϕ′(u)hc\phi'(u)h. Whether pushing u≤τu\le\tau persists under multi-step AdamW is an empirical question we now measure directly.

  2. Gate derivative damping persists across 30 epochs: Tracking the 308 selected channels across 4 layers on 4 fixed examples (all weights trainable, no attack-time sealing/hooks):

    Table B.6: Mean abs. SiLU'(u) on selected channels across 30 epochs.

    EpochGSUGSU w/o seal (2× suppression)GSU drop (%)
    0 (Release)0.3016310.39220823.09
    100.3721770.42202911.81
    200.3878860.43668111.17
    300.3915460.44227311.47

    After early adjustment, the relative derivative reduction stabilizes at 11.17%–11.81% through epoch 30.

  3. Actual AdamW steps alongside damped gate responses:

    Table B.7: Actual parameter update RMS (×10⁻⁸) across 390 AdamW steps.

    ProjectionGSUGSU w/o seal (2× suppression)Ratio
    Gate6.0775.6171.08
    Up7.5876.5891.15
    Down8.2497.2231.14

    Because AdamW normalizes gradients via second moments, GSU's parameter steps are slightly larger (1.08×–1.15×). Thus, the lasting effect comes from persistent gate-response damping in the SiLU tail (Table B.6), which makes those weight updates yield 11.4% less capability acquisition from a matched start (Table B.2) and lower UA90@30 (Table B.1). We will add Tables B.6–B.7 to make this explicit.

AuthorsOpenReview ↗

Response to Reviewer yHEn (Part 3/4)

Weakness 4: Coverage of the stated attacker threat model

“the experimental evaluations are relatively narrow compared with the threat model considered in the paper. The authors assume an attacker with full access to the released model parameters, who can choose the finetuning objective, optimizer, schedule, and training configuration. However, the implemented evaluation mainly considers standard supervised finetuning with a fixed optimization procedure.”

Under our threat model, the attacker has full white-box access to θrel\boldsymbol{\theta}_\text{rel} with no defender hooks or constraints. To test broader attack strategies, we evaluated Qwen3.5-2B (10 epochs / 130 updates) across 3 constant AdamW LRs, warmup+cosine, Momentum SGD, LoRA, and equal-budget 5-way attack selection per defense (including 2-candidate pilots for SGD/LoRA):

Table B.8: Attack sensitivity across optimizers, schedules, LRs, and LoRA (%).

AttackGSU UA90@10 ↓NPO+SAM UA90@10 ↓
AdamW, 10⁻⁶4.9815.312
AdamW, 3×10⁻⁶10.39810.454
AdamW, 10⁻⁵84.17628.507
10% warmup + cosine75.28636.949
Momentum SGD6.4135.533
LoRA22.40312.639

GSU beats NPO+SAM at default (10⁻⁶) and 3× higher (3×10⁻⁶) AdamW LRs, and keeps recovery low under Momentum SGD (6.413%). In the 5-way search, AdamW 10⁻⁵ is selected for both defenses (30-epoch UA90: 84.18% vs. 28.51%; max across all 6 configs: 84.18% vs. 36.95%). Under such aggressive LRs or LoRA, the ranking flips to NPO+SAM: large steps push pre-activations back across τ\tau and LoRA bypasses sparse G\mathcal{G}, where SAM's global flatness helps—pointing to GSU+SAM as a natural extension. We will add Table B.8 to clearly mark these operating boundaries.


Weakness 5: Sensitivity to the optimizer, schedule, and defense-specific tuning

“The attack experiments use has a fixed learning rates and no warmup. It is unclear whether the proposed gradient attenuation would remain effective under other optimizers, learning-rate schedules, or hyperparameters optimized specifically against GSU.”

Addressed directly in Weakness 4 (Table B.8): testing alternative optimizers, warmup+cosine schedules, LoRA, and equal-budget worst-case tuning shows GSU keeps lower UA90@10 at 10⁻⁶ and 3×10⁻⁶ AdamW LRs (and low recovery under Momentum SGD, 6.413%), while explicitly charting the high-LR (10⁻⁵, warmup+cosine) and LoRA regimes where the ranking reverses.


Weakness 6: Generalization beyond proxy entities

“The experiments evaluates disjoint datasets but the separation is still limited. These setup does not show whether the selected gradient pathways generalize to very different datasets with different distributions of forbidden knowledge. I think a setup where an attacker tries to fine-tune and acquire a completely different unsafe capabililty is more realistic.”

To test transfer beyond new facts about the same authors, we built an entity-disjoint TOFU split on Qwen3.5-2B: defense uses 10 target authors (200 QA pairs), while the attacker fine-tunes on 10 completely unseen authors (200 QA pairs) absent from localization, training, and calibration.

Table B.9: Acquisition on entirely unseen target authors (%).

MethodES₀ ↓ES₃ ↓ES₅ ↓UA90@10 ↓
GSU5.1615.5935.9606.265
NPO+SAM5.6105.7916.0096.299
No Defense6.1856.5196.7177.148

GSU cuts UA90@10 from 7.148% to 6.265% (12.4% reduction) and outperforms NPO+SAM at every stage, showing the localized gates capture shared domain pathways rather than entity names. Blocking a completely unrelated capability outside the proxy domain lies beyond domain-designated preemptive unlearning (§2)—indiscriminate sealing would conflict with preserving benign fine-tunability (Fig. 4c).

Reviewer_tFAkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies preemptive unlearning: modifying model weights before release to limit subsequent acquisition of a designated capability while preserving ordinary utility. Gradient-Sealed Unlearning (GSU) trains a temporary model copy on proxy data, identifies MLP gates with upward pre-activation shifts, then returns to the original weights and jointly optimizes a sealing penalty, response suppression and retention. A local analysis motivates distinguishing release-time outputs from susceptibility to future updates. Experiments evaluate knowledge-absent prevention on TOFU across six Llama/Qwen models and hazardous-knowledge reacquisition on WMDP, supplemented by gate measurements, a sealing ablation and benign fine-tuning analyses.

Critical strengths
  1. A concrete intervention built around a meaningful distinction. The expose–localize–reset–seal procedure uses proxy learning to identify gates, then discards the proxy-trained weights. The motivating study reports a correlation between the activation-change rankings under proxy learning and attack (rho = 0.617). Targeted attenuation during acquisition also reduces acquisition gain by up to 24.35%. These findings, together with the local analysis, give a plausible rationale for targeting learning pathways rather than focusing only on outputs at release (Sections 3–4, pp. 4–7; Appendix C, pp. 17–19).

  2. Useful comparisons with explicit utility constraints. The main study covers six TOFU models and two WMDP domains, calibrates utility at release, and reports a utility-qualified acquisition metric. In the disjoint setting for Llama-3.2-1B, for example, GSU reduces UA90 to 27.87%, compared with 30.52% for the strongest baseline and 50.08% without defense. The paper also reports convergence late in the attack and mixed results for benign accuracy, making the limits of the practical benefit clearer (Tables 1–2, p. 8; Section 5.3, p. 9; Appendix F).

Critical weaknesses
  1. The evidence for the mechanism is weaker than the abstract’s claim of blocking acquisition pathways suggests. In the paired sealing ablation, adding the sealing term gives an ES advantage of 2.858 percentage points at release, 0.934 at attack epoch 5, and 0.061 at epoch 10 (Appendix F.2, p. 24). This shows a benefit under an early attack budget, but because the models start with different scores, the ablation cannot isolate a reduction in subsequent learning. The authors appropriately acknowledge this limitation. The paper also reports lower local SiLU derivatives at release and a separate intervention that attenuates updates during the attack. These results do not establish that the released GSU weights maintain this attenuation under ordinary full-model AdamW updates. A suppression control matched on release ES and utility, or measurements of the relevant updates during acquisition, would help distinguish the explanations. The current diagnostics provide support for the proposed mechanism, but they do not establish that it causes the gains in the main tables (Section 5.3; Appendices F.2 and F.5).

  2. The main comparisons lack enough detail to assess reproducibility and robustness. Appendix E describes the shared training recipe, but it does not list the GSU suppression and retention losses used in the main runs, their weights, the gate-selection settings, or the selection procedure. Table 4 gives settings only for the separate further analyses, whose configurations were selected using attack evaluations. Those diagnostics therefore need to be clearly distinguished from the six-model main comparison. The main runs also use seed 0 and one fixed attack recipe per model, so the small differences and rank counts do not establish stability across runs or attack settings (Tables 1–2; Appendix E.5 and the opening of Appendix F, p. 22). The missing configuration details are the immediate concern. After those are provided, a limited sensitivity analysis on the defended checkpoints would strengthen the broader claim of resistance.

Questions for authors

Could you provide the exact GSU loss definitions, hyperparameters, checkpoint initialization, and selection procedure used for Tables 1–2? Please also explain how those runs relate to the separately configured diagnostics in Figure 4 and Appendix F.

7 comments
AuthorsOpenReview ↗

Response to Reviewer tFAK (Part 1/3)

Thank you for recognizing the distinction between release-time suppression and acquisition resistance, and for pinpointing the controls needed to isolate our mechanism. We address each point below and will add all new results and configurations to the revision.


Weakness 1: Separating initial suppression from acquisition resistance

“The evidence for the mechanism is weaker than the abstract’s claim of blocking acquisition pathways suggests. In the paired sealing ablation, adding the sealing term gives an ES advantage of 2.858 percentage points at release, 0.934 at attack epoch 5, and 0.061 at epoch 10 (Appendix F.2, p. 24). This shows a benefit under an early attack budget, but because the models start with different scores, the ablation cannot isolate a reduction in subsequent learning. ... A suppression control matched on release ES and utility, or measurements of the relevant updates during acquisition, would help distinguish the explanations. The current diagnostics provide support for the proposed mechanism, but they do not establish that it causes the gains in the main tables (Section 5.3; Appendices F.2 and F.5).”

You make a fair point: when two models start at different release scores, a lower post-attack curve alone cannot prove slower learning. To isolate the effect of sealing on Qwen3.5-2B/TOFU (disjoint), we (1) match a no-seal control on release ES and utility (showing 11.4% more acquisition without sealing over 30 epochs), (2) extend the attack 3× to 30 epochs against doubled unsealed suppression (showing harder release suppression rebounds faster), and (3) track gate derivatives and AdamW updates directly (confirming an 11.47% lower gate derivative persists through epoch 30).

  1. A release-matched control confirms slower acquisition from the same start

    From saved no-seal runs, we calibrated a control using one coefficient chosen solely on defender data (B1 ES and retain utility) to match GSU within ±0.05 pp on release ES and ±0.20 pp on release utility, froze it prior to attack, and ran the same 30-epoch (390-update) full-parameter AdamW attack.

    Table A.1: Release-matched 30-epoch comparison (%).

    MethodES₀ES₃ES₅ES₃₀UA90@30
    GSU3.4563.9254.7065.9385.944
    GSU w/o seal (matched release)3.4424.1284.9676.2076.213

    Both models start together (ES₀ gap 0.014 pp: 3.456% vs. 3.442%; utility 100.18% vs. 100.31%—slightly favoring the control on both), yet GSU stays lower at every stage. Over 30 epochs, the no-seal control gains 2.765 pp vs. 2.482 pp for GSU (11.4% more acquisition without sealing) and ends 0.269 pp higher in UA90@30. This follows our standard protocol, and the same direction holds across all three attack seeds in Table A.8.

  2. Stronger unsealed suppression rebounds faster over a 3× longer attack

    Next, we tested whether amplifying output suppression replaces sealing over 30 epochs (390 updates): GSU w/o seal drops only sealing, while GSU w/o seal (2× suppression) doubles the NPO weight (selected on defender scores before attack).

    Table A.2: Release scores and 30-epoch recovery (%).

    MethodES₀ES₃ES₅ES₃₀UA90@30
    GSU3.4563.9254.7065.9385.944
    NPO+SAM4.4554.6924.9486.1406.140
    GSU w/o seal5.0345.1155.2946.3786.378
    GSU w/o seal (2× suppression)4.5265.0905.4426.6796.679

    GSU remains lowest across the 3× budget (UA90@30 5.944% vs. 6.140% / 6.378%, retaining 98.05% utility). Doubling NPO lowers release ES₀ (5.034% → 4.526%) yet raises UA90@30 (6.378% → 6.679%): forcing outputs down without sealing internal gates leaves pathways prone to sharper rebound.

  3. Gate attenuation persists across 390 ordinary AdamW updates

    Tracking the selected channels on 4 fixed target examples during all 390 full-model AdamW updates—with every weight trainable and no attack-time sealing or hooks—confirms that the gate shift endures:

    Table A.3: Gate sensitivity and parameter updates during attack.

    MeasurementEpoch / IntervalGSUGSU w/o seal (2× suppression)
    Mean abs. SiLU'(u)Release (0)0.3016310.392208
    Mean abs. SiLU'(u)Epoch 100.3721770.422029
    Mean abs. SiLU'(u)Epoch 200.3878860.436681
    Mean abs. SiLU'(u)Epoch 300.3915460.442273
    Gate / Up / Down RMS (×10⁻⁸)All 390 updates6.077 / 7.587 / 8.2495.617 / 6.589 / 7.223

    Even after 30 epochs, GSU keeps an 11.47% lower mean |SiLU'(u)| (0.391546 vs. 0.442273). Because AdamW normalizes steps by second moments, GSU's weight updates are slightly larger, not smaller; yet staying in the low-slope negative tail translates those steps into 11.4% less capability gain from a matched start (Table A.1).

Overall, GSU's benefit holds from a matched starting state, persists over a 3× longer attack, and coexists with sustained gate attenuation under ordinary AdamW updates.

AuthorsOpenReview ↗

Response to Reviewer tFAK (Part 2/3)

Weakness 2: Configuration disclosure and robustness across runs and attacks (Part 1/2)

“The main comparisons lack enough detail to assess reproducibility and robustness. Appendix E describes the shared training recipe, but it does not list the GSU suppression and retention losses used in the main runs, their weights, the gate-selection settings, or the selection procedure. Table 4 gives settings only for the separate further analyses, whose configurations were selected using attack evaluations. Those diagnostics therefore need to be clearly distinguished from the six-model main comparison. The main runs also use seed 0 and one fixed attack recipe per model, so the small differences and rank counts do not establish stability across runs or attack settings (Tables 1–2; Appendix E.5 and the opening of Appendix F, p. 22). The missing configuration details are the immediate concern. After those are provided, a limited sensitivity analysis on the defended checkpoints would strengthen the broader claim of resistance.”

We address reproducibility and robustness by (1) documenting the shared protocol and exact main-table GSU losses, hyperparameters, and selection rules (Tables A.4–A.6), (2) separating them from the frozen Figure 4 / Appendix F diagnostics (Table A.7), and (3) adding 3-seed and 6-optimizer sensitivity tests on the defended checkpoints (Tables A.8–A.9), showing consistent gains across seeds and standard/moderate AdamW rates while delineating high-LR and LoRA boundaries.

  1. Common comparison protocol

    Within each model, all methods share the same base checkpoint, data split, and attack (TOFU-disjoint uses separate records on the same authors; TOFU-identical uses the same records).

    Table A.4: Common comparison protocol.

    ComponentShared Protocol
    TrainingSeed 0; BF16; max seq len 512; effective batch size 16
    TOFUShared retain-only base; 10-epoch (130-update) defense & attack
    WMDPOriginal Zephyr-7B-β; 80-update defense & 150-update attack
    Attack LRCalibrated once per model on No Defense, fixed across all methods
    UtilityFixed UrefU_\text{ref} (retain-only base / original model; TOFU = HM(MU, Fluency), WMDP = MMLU); UWC release ≥95%; UA90 attack bar ≥90%
  2. Exact GSU losses, localization, and model-specific settings

    GSU optimizes LGSU=Lsup+λsLseal+λrLret\mathcal{L}_ \text{GSU}=\mathcal{L}_ \text{sup}+\lambda_ s\mathcal{L}_ \text{seal}+\lambda_ r\mathcal{L}_ \text{ret}, using NPO (β=0.1\beta=0.1, weight 1) for Lsup\mathcal{L}_ \text{sup}, retain NLL (λr=1\lambda_ r=1; answer tokens on TOFU, valid corpus tokens on WMDP) for Lret\mathcal{L}_ \text{ret}, and Lseal(θ;Df)=E(x,y)∼Df[1∣G∣∣y∣∑(ℓ,j)∈G∑i=1∣y∣max⁡(uℓji(θ)−τ,0)2],\mathcal{L}_ \text{seal}(\theta;\mathcal{D}_ f)=\mathbb{E}_ {(x,y)\sim\mathcal{D}_ f}\big[\frac{1}{|\mathcal{G}||y|}\sum_ {(\ell,j)\in\mathcal{G}}\sum_ {i=1}^{|y|}\max(u_ {\ell j}^i(\theta)-\tau,0)^2\big], where G\mathcal{G} is the selected gate set and uℓjiu_{\ell j}^i is the answer-token pre-activation (Lseal=0\mathcal{L}_\text{seal}=0 if G=∅\mathcal{G}=\emptyset). Localization runs 100 lookahead steps (LR 5×10⁻⁶) on a temporary copy to score upward pre-activation shifts; we then discard that copy and train from the shared base θo\theta_o via first-order updates.

    Table A.5: GSU settings for new Qwen3.5-2B control runs (W1, W2.4–2.5).

    ComponentSetting
    LossesNPO (β=0.1\beta=0.1, w=1); Answer NLL (λr=1\lambda_r=1); Seal (λs=0.03\lambda_s=0.03, τ=−4\tau=-4)
    Localization100 updates at LR 5×10⁻⁶; K=4K=4 layers; p=5%p=5\% channels/layer
    Release & AttackPasses ≥95% utility without UWC mixing; default AdamW LR 10⁻⁶

    Table A.6: Model-specific GSU settings in Tables 1–2.

    SettingModelKKppλs\lambda_sτ\tau
    TOFU / disjointLlama-3.2-1B41%0.01-6.262
    Llama-3.2-3B15%0.003-9
    Llama-3.1-8B25%0.003-4
    Qwen3.5-2B45%0.01-9
    Qwen3.5-4B45%0.01-6.262
    Qwen3.5-9B25%0.003-6.262
    TOFU / identicalLlama-3.2-1B110%0.003-9
    Llama-3.2-3B15%0.01-4
    Llama-3.1-8B25%0.003-6.262
    Qwen3.5-2B21%0.003-9
    Qwen3.5-4B25%0.003-9
    Qwen3.5-9B25%0.003-6.262
    WMDP-Bio / disjointZephyr-7B-β25%0.01-6.262
    WMDP-Cyber / disjointZephyr-7B-β25%0.01-6.262

    Selection procedure: Across all models, we fix NPO, NLL, and lookahead (Table A.5), sweep only (K,p,λs,τ)(K,p,\lambda_s,\tau), filter for ≥95% release utility, and pick the candidate with the lowest target metric (ES₅ on TOFU, F₅ on WMDP). Main-table and diagnostic records are kept strictly separate.

AuthorsOpenReview ↗

Response to Reviewer tFAK (Part 3/3)

Weakness 2: Configuration disclosure and robustness across runs and attacks (Part 2/2)

  1. Separately frozen diagnostic settings (Figure 4 & Appendix F)

    Figure 4 and Appendix F use the frozen checkpoints in Table A.7 (Table 4 in the paper), selected early by lowest mean ES over epochs 1–5 among completed disjoint candidates at freeze time (defense LR 10⁻⁵, 100 lookahead steps, mixing coeff 1; baselines use diagnostic defaults), so F.1–F.5 analyze one consistent run per model.

    Table A.7: Frozen diagnostic configurations.

    ModelKKppλs\lambda_sτ\tau
    Llama-3.2-1B45%0.01-9
    Qwen3.5-2B45%0.03-4
    Llama-3.2-3B25%0.01-4
  2. Consistency across three paired attack seeds

    Repeating the 10-epoch attack with seeds 0, 1, 2 on fixed Qwen3.5-2B GSU and NPO+SAM releases (seed 0 = first 10 epochs of Table A.2) shows consistent gains:

    Table A.8: Three paired attack seeds (%).

    SeedGSU ES₃NPO+SAM ES₃GSU ES₅NPO+SAM ES₅GSU UA90@10NPO+SAM UA90@10
    03.9254.6924.7064.9484.9815.312
    13.9624.7174.7064.8955.1535.350
    24.0224.7314.6764.9764.9815.351
    Mean±SD3.970±0.0494.713±0.0194.696±0.0184.940±0.0415.038±0.0995.338±0.022

    GSU beats NPO+SAM on every seed and metric, reducing ES₃, ES₅, and UA90@10 by 0.744, 0.244, and 0.300 pp on average.

  3. Sensitivity across attacker optimizers, schedules, and LoRA

    We varied the attacker's LR, schedule, optimizer, and parameterization on Qwen3.5-2B (10 epochs) under equal tuning budgets:

    Table A.9: Sensitivity to attacker optimization choices (%).

    AttackGSU UA90@10 ↓NPO+SAM UA90@10 ↓
    AdamW, 10⁻⁶4.9815.312
    AdamW, 3×10⁻⁶10.39810.454
    AdamW, 10⁻⁵84.17628.507
    10% warmup + cosine75.28636.949
    Momentum SGD6.4135.533
    LoRA22.40312.639

    GSU holds its advantage at default (10⁻⁶) and 3× higher (3×10⁻⁶) AdamW rates and stays low under Momentum SGD (6.413%). At 10⁻⁵ (worst-case 5-way pick for both defenses; 30-epoch UA90 84.176% vs. 28.507%), warmup+cosine, or LoRA, the ranking flips to NPO+SAM: large steps can push pre-activations back across τ\tau and LoRA can route around sparse G\mathcal{G}, where SAM's global flatness helps. Reporting Table A.9 makes both the effective regime and these boundaries explicit.


Question 1: Exact losses, initialization, selection, and diagnostic configurations

“Could you provide the exact GSU loss definitions, hyperparameters, checkpoint initialization, and selection procedure used for Tables 1–2? Please also explain how those runs relate to the separately configured diagnostics in Figure 4 and Appendix F.”

Yes, provided in Weakness 2 (Items 1–3, Tables A.4–A.7). Crucially, lookahead only selects which gates to seal; defense training always restarts from the same pre-defense base θo\theta_o as all baselines, and Table 1–2 configs are kept strictly separate from the frozen diagnostics in Table A.7.

Comments on the paper

1 comment
Area_Chair_HxYfOpenReview ↗

Follow-Up on the Authors’ Rebuttal

Dear reviewers,

The authors have posted their rebuttal. Please take a moment to read their response and leave a brief follow-up comment on whether it addresses the concerns you raised in your review. If any points remain unclear or you have follow-up questions, please raise them during the rebuttal period so the authors have a chance to respond. Please also indicate whether the rebuttal changes your overall assessment of the paper.

Thank you !

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.