acceptodds
Under review as a conference paper at ICLR 2027

Attention-Only Machine Unlearning in Large Language Models

Abstract

Machine unlearning aims to remove designated knowledge from a model while preserving its utility and minimizing computational cost. However, existing approaches often struggle with *knowledge entanglement*, where targeted and retained knowledge share model representations. In this work, we investigate attention-only unlearning for large language models (LLMs), focusing on a component that is central to the Transformer architecture yet often overlooked as a target for unlearning. Inspired by recent insights into memory retrieval through attention mechanisms, our method systematically identifies context-specific "anchored terms” and confines parameter updates to attention layers to enable targeted, granular unlearning. Experiments show that our approach achieves unlearning effectiveness competitive with existing techniques while preserving performance on retained data and reducing computational requirements. These efficiency gains stem from updating attention modules, which contain fewer parameters than the corresponding multilayer perceptron (MLP) modules in the evaluated architectures. Our findings highlight attention as a promising target for efficient unlearning and offer insights into its role in selectively accessing and suppressing knowledge in LLMs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model, Machine unlearning, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CXVgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes attention-only unlearning for LLMs. They train an "Anchored Terms Identifier" (ATI) (a small MLP classifier trained on features of entity-sensitive attention states) in order to put more weights on subject/entity tokens in forget inputs. Subsequent unlearning defines loss over model internal states instead of final output, which is different from many existing unlearning algorithms.

Critical strengths
  • Validated interpretability-grounded design. The core design choice of targeting ATI-selected anchored tokens is useful for interpretability and validated with a downstream selection ablation where ATI beats frequency/random/whole-input targeting in 5 out of 6 model–dataset pairs (Table 4). This directly supports the knowledge-entanglement motivation.

  • Large efficiency and retention gains. Updating only K/V in five layers yields the best retention scores on SUITE across all models (Table 2) at a fraction of the compute (Sec. 3.5).

Critical weaknesses
  1. The headline win depends on the authors' custom aggregate, and per-metric forgetting is weak. On SUITE, "Ours" is not best at forgetting (F=0.68 vs NPO 0.71 on Llama, Table 2); on TOFU it leads no individual metric (Table 3). If we change the weights of different individual metrics, the overall ranking might be different. More concerning, EM score on SUITE remains 2–5× higher than NPO (e.g., 0.346 vs 0.073 on Phi-4, Table 7), and INT4 quantization partially reverses forgetting on TOFU (Fig. 3). This makes it unsuitable for the authors' critique of existing works in related work: "suppressing the expression of knowledge does not necessarily eliminate its underlying influence." In addition, no relearning attack is run to verify whether information is actually eliminated or not.

  2. The paper is structured and written in a somewhat confusing way for general readers, leaving some design choices unexplained.

  • Eq (1), what's x? And how does it relate to the right hand?
  • Line 198: what's k? Please define the ML task more clearly, otherwise it is difficult to understand these hyperparams and the numbers in in Table 1.
  • How to read figure 2? what does a bar mean?
  • Eq (3): why iterating over R_x? And please explain this loss: why is it valid by minimizing the attention score?
  • Line 300: why exclude identified anchored spans?
Questions for authors

In addition to questions in the weakness:

  • Why is EM on SUITE much higher than NPO (Table 7)? How does the method behave under benign relearning or stronger extraction attacks — does knowledge remain recoverable?
  • For SUITE, are the steering reference tokens still TOFU author names?
  • Line 431: why Qwen 2.5 on SUITE is the exception?
3 comments

We thank the reviewer for their insightful reviews.

We would like to address your concerns and questions in 2 parts

PART 1:

  • Weakness 1: Unlearning results reported in the paper are for 1 seed with both open unlearning baselines and our own method remain largely untuned (as mentioned by reviewer LT8i and iZTT). As a results, we have added a 3 seeds run as well as tuning for all methods in the experiment. Table 2, 3 are updated with the new results. Standard deviation reported for Overall metric. To summarize, our method achieving best unlearning results in overall for 5/6 model-dataset, with the remaining TOFU Phi-4 being top 2 (0.04 behind UNDIAL). We selected the weights for FF, DD, and GG to reflect the two equally important unlearning objectives of:
    • Forget effectiveness (FF)
    • Retention: DD - retention on retain dataset, and GG - retention on general capabilities dataset. It is indeed possible for other methods to be best if we change the weights of the 3 metrics in the harmonic mean calculation, but that would reflect a different objective that might weight forgetting more or retention more.
  • Weakness 2:
    • xx here in Equation 1 is the input. An input xx is made up of many tokens tt. Equation 1 essentially reads "For all non-padded token tt in input xx, select all tt with sigmoid score ptp_t greater than or equal to a threshold τ\tau". Where sigmoid score ptp_t is assigned to tokens by our ATI module.

    • kk in line 198 is further described in Table 1's caption: "@k@k denotes the output cutoff". To evaluate ATI's performance, we select only top kk scored tokens by ATI for evaluation.

    • Figure 2 shows the causal probing results of ATI versus random subject tokens (left) and versus random non-subject tokens (right). X axis shows Contrastive Impact, a metric defined above (line 208-210) that essentially show how much corrupting certain tokens weakens the model's preference for the correct factual answer over an incorrect counterfactual answer.

      For example: Input - What is the capital of France ? Factual answer: Paris Counterfactual answer: London Under a clean run, assuming the model has the proper knowledge to answer the question, log-prob of Paris should be higher than London. Under a corrupted run that corrupts different tokens, we show that out ATI identifies tokens that results in larger decrease to factual answer (Paris). So causally, ATI's identified tokens are what yield the correct answer, while corrupting them does the opposite.

      Not only that, since a term like "France" could be tokenized as 2 sub-word tokens "Fr" and "ance" (hypothetically), which of the 2 tokens are more important to the factual recall process ? Left figure shows that our ATI can also identify the more important sub-word tokens that are essential to factual recall.

      Comparison in Figure 2 take the differences between CI produced by ATI's tokens and CI produced by random tokens. A positive number means ATI has better CI than random's.

    • Iteration over RxR_x here represents our objective of disrupting attention to subject tokens from non-subject tokens (which is RxR_x) instead of from subject tokens with each other.

      For example: Input - "What is the capital of France and China ?", with "Fr" "ance" and "Chi" "na" being the subject tokens (again, assuming a sub-word tokenization).

      When processing token like "capital" (in RxR_x), it might assign high attention to tokens in "France" and "China". This unlearning objective reduce this attention value.

      We purposely exclude subject tokens' attention to themselves on purpose, under the intuition that doing so might change model's knowledge about the subjects themselves, instead of a fact (capital) regarding the subjects.

    • In line 300, we mention the exclusion of identified anchored spans in our Retain-projection preservation. This is due to the fact that forget data and retain data often contain the same subjects, but different facts. For example: SUITE dataset's forget data contains input querying "The 1986 Space Shuttle Challenger disaster", while retain data contains "The rest of Space Shuttle history". Same subject "Space Shuttle", but different contexts. If we include the identified anchored spans during the retention loss, we argue that it will unwind forget loss imposed on the identified anchored during forget loss.

AuthorsOpenReview ↗

Rebuttal part 2

PART 2:

  • Questions:
    • "Why is EM on SUITE much higher than NPO (Table 7)? How does the method behave under benign relearning or stronger extraction attacks — does knowledge remain recoverable?"

      Table 7 is now table 9, EM is a metric that we expect to underperform compared to other methods due to our loss objective. Methods like NPO that train directly on forget data labels will push the log-probability of label tokens down. This decrease in log-prob of label token is what EM measures.

      On the other hand, our loss objective mainly concerns representations of identified subject tokens, which does not directly change log-prob. However, this is not necessary a downside, as we see that despite the higher EM, Forget ROUGE remains low, so no sensitive information is actually leaked.

      We have included a relearning attack influenced by https://doi.org/10.48550/arXiv.2406.13356](https://doi.org/10.48550/arXiv.2406.13356). The attack relies on benign samples that are relevant to the unlearn data without the explicit knowledge expression.

      For SUITE, we selected a subset of 100 samples each from Syntactic and Lexical in Retain Eval split. Syntactic includes samples that share the same question structure as forget data, while Lexical includes samples that share the same term (term is the unlearning subject in question) as forget data but different context.

      For TOFU, it is more difficult to find a dataset with the same forget authors set but different context. So we opted for the Real Authors dataset included in TOFU.

      The attack finetunes the selected dataset in 5 epochs with a fixed learning rate of 1e-05. We perform the attack on both our own method and baseline methods. Due to the large comparison, we have composed the results in a graph to be included in the Appendix A.7.

      To summarize, our method generally maintain unlearning effectiveness across models and datasets as reported by the 3 unlearning metrics, the only exception is Qwen2.5 on SUITE dataset where a slight increase to ES (0.55 --> 0.65) and EM (0.2-->0.4) are observed while ROUGE metric maintain low after 5 epochs. We also want to note that Qwen2.5 SUITE has only 3 layers targeted by our unlearning engine. It is possible that the increase in EM and ES are due to this smaller number of modified layers, leading to other retrieval pathway in Attention. However, such increases in EM and ES do not directly yield the actual leakage of sensitive information itself, as seen by ROUGE metric.

    • "For SUITE, are the steering reference tokens still TOFU author names?"

      Yes, steering for SUITE also uses the TOFU fictitious author names. The reason why we keep it this way is due to the fact that these are fictitious author names, they do not exist in real life and have never been seen by the models.

      So theoretically, these fictitious names are treated as "unknown" combination of tokens with little to no significant semantic.

      For example: a subject in SUITE might be "Space Shuttle", which is an object rather than someone's name, steering it towards "Siriporn Nakharin" (fictitious name) might carry over unwanted semantic. But since the name itself is never seen by models, from the model's perspective (theoretically), they could be anything.

AuthorsOpenReview ↗

Summary of changes to the document

  1. Table 2, 3 updated with new 3 seeds run and tuning results. λkv\lambda_{kv} descriptions in Table 2 and 3's captions are removed. Section 3.4.1 is modified to reflect the new results.
  2. Tables 9, 10, 11, 12 in Appendix A.6 are updated with the new seeded runs and tuning results.
  3. Table 4 updated with the new tuned result on first seed only. We also added comparison with anchored selected by spaCy NER.
  4. Figure 4 updated with total wall-clock and per-optimizer-step time. Reporting only relative speedup compared to mean baseline's speed. Relevant sections are also updated to reflect new results from Figure 4 (Section 3.5, Section 4, Section 1 - last paragraph).
  5. Update Equation 1 and Section 3.2 (first paragraph) to not explicitly determine a subject token selection threshold τ=0.75\tau=0.75
  6. Update Appendix A.5 to include detailed description and analysis of the tuning process as well as documented configurations.
  7. Add Appendix A.7 Relearning attack that shows details about the relearning attack.
  8. Add Appendix A.8 Unlearning objective ablation. Here, we ablate different components of out unlearn loss objective (Equation 2)
Reviewer_LT8iOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes an LLM unlearning method that updates only the key and value projections of a few attention layers, and focuses each update on "anchored terms": lexical cues such as entity names.

  • Anchored Terms Identifier (ATI). A small MLP trained on Mintaka entity spans scores each token from four attention-derived features of entity-sensitive heads. Tokens scoring at least 0.75 become anchors.
  • Attention Unlearning Engine. It combines four losses:
    • a QK-disruption loss that penalizes attention from non-anchor queries to anchor tokens;
    • a steering loss that pulls anchor keys (and optionally values) toward those produced by fictitious TOFU author names;
    • an MSE loss that keeps K/V projection outputs unchanged on retain inputs;
    • a standard retain loss.
  • Layer selection. Five target layers per model are chosen by key-only activation patching on CounterFact.
  • Evaluation. Experiments use TOFU and SUITE with Llama-3.1-8B-Instruct, Qwen2.5-14B-Instruct and Phi-4, against NPO, SimNPO, SatImp, UNDIAL and RMU.
  • Reported results. The method achieves the best custom "Overall" score in 5 of 6 settings and 4.3–32× faster optimizer steps. The paper also includes a quantization stress test and an anchor-selection ablation.
Critical strengths
  1. Well motivated and cheap. Restricting updates to attention K/V and to cue tokens is a clean hypothesis. It fits interpretability evidence that attention moves subject information to later positions, and it touches only 0.36–1.34% of parameters.
  2. Good experimental breadth. The study covers:
    • three recent instruction-tuned models;
    • two benchmarks, including SUITE's paraphrase and lexical-overlap retain sets;
    • five baselines run through OpenUnlearning;
    • seven general-capability benchmarks;
    • a quantization attack.
  3. Strong retention on SUITE. Retain-set performance (D) is the best of all methods for all three models (Table 2). The lexical and Tier-0 subsets give some evidence that editing an entity's keys does not break benign uses of that entity.
  4. ATI is validated causally, not only by tagging accuracy. The noise-ablation probe (Fig. 2) shows that corrupting ATI-selected tokens reduces factual preference more than corrupting random subject tokens. The ATI-minus-random contrastive-impact gaps are 0.49 / 0.29 / 0.23 across the three models.
Critical weaknesses

W1. The efficiency claim is misstated, and the comparison is not like-for-like.

  • Misstated. The contributions claim "4–32× fewer optimizer steps", but Table 6 shows every method trains for 10 epochs at effective batch size 32, so the number of steps is identical. Section 3.5 and Fig. 4 measure wall-clock time per step, which is a different claim.

  • Not like-for-like. The speedups split cleanly by configuration:

    • On Llama, where all methods use the same micro-batch (4 × 8), the speedup is 4.3–4.6×.
    • On Qwen and Phi-4, the baselines were run at micro-batch 1 × 32 to fit in memory, and the speedup is 16.7–32.1×.

    Much of the headline number therefore reflects micro-batch size and full-model training, not the method itself.

  • Missing costs. ATI training, head analysis and causal-patching layer selection are not counted. These are one-time costs per model, but they should be reported.

  • Missing baseline. There is no parameter-matched baseline, such as NPO, SimNPO or RMU restricted to the same K/V parameters, or LoRA with a matched trainable budget.

W2. It is unclear what drives the results.

  • No component ablations. There is no ablation of the four loss terms, of K vs. V vs. K+V updates, or of keeping Q frozen.
  • Location vs. objective not separated. The thesis is that the location (attention) matters. Yet the objectives are never applied to MLP layers, and standard objectives are never applied to the same attention parameters. Either experiment would separate location from objective.
  • The selection ablation is mixed.
    • Random selection beats ATI on Qwen/SUITE (Overall 0.586 vs. 0.505).
    • Random or "All" selection often forgets more, with lower forget ROUGE and EM in several cells of Table 4.
  • Weak comparison point. ATI is an entity-span classifier trained on Mintaka, so an off-the-shelf NER tagger is the natural baseline. Frequency-based selection is a weak substitute.

W3. The TOFU protocol and the robustness evaluation are incomplete.

  • TOFU setup. The forget split (forget01/05/10) is not stated, there is no retain-only reference model, and the standard TOFU forget-quality statistic is not reported.
  • Variance and tuning. All results are single runs with no variance estimate. λ_kv is chosen per model and dataset (5 vs. 6) on the reported benchmarks themselves.
  • Robustness. The only attack is quantization, which partially reverses forgetting on TOFU at INT4 (EM up to +7 points). There is no relearning attack and no probing for residual knowledge.
  • Anchor-avoiding queries. For a cue-based method, the central threat is retrieval through queries that avoid the anchor term. Examples are descriptive references ("the author of ⟨book⟩"), pronouns in multi-turn context, and misspelled or transliterated names. None of these is tested.
Questions for authors
  1. Please correct or justify "4–32× fewer optimizer steps."
    • Can you report per-step and total wall-clock time against baselines run at the same micro-batch size?
    • Please also include NPO or RMU restricted to the same K/V parameters, or LoRA with a matched trainable budget.
    • Please include the ATI training and layer-selection costs.
  2. How do results change when each loss term is removed? With K-only, V-only and K+V(+Q) updates? And when your objectives are applied to the MLP blocks of the same five layers?
  3. Which TOFU forget split is used? Can you report TOFU forget quality against a retain-only model, and mean ± std over at least 3 seeds?
  4. How robust is the forgetting to:
    • (a) a relearning attack (brief fine-tuning on related data);
    • (b) queries that avoid the anchored term (descriptions, pronouns in multi-turn context, misspelled names)?
  5. What explains the large IFEval, GSM8K and HumanEval drops for Llama on SUITE, and the 69% gibberish rate on TOFU/Llama? Is there a setting that keeps general capability near the baselines while still forgetting?
  6. How does ATI compare with an off-the-shelf NER tagger as the anchor selector?
2 comments

We thank the reviewer for their insightful reviews.

We would like to address your concerns and questions as followed:

  • Weakness 1/Question 1: "4–32× fewer optimizer steps" this is indeed misstated, what should be stated is "4-32x speedup optimizer step time".

    The speedup difference in Llama versus Qwen and Phi-4 is due to the model size differences. Since our method target 5 layers for unlearning despite model size, Llama, with an 8B size, has more trainable parameters compared to Qwen and Phi-4, which have 14B parameters.

    In the new figure 4, only relative speedup is reported now but in addition to the per optimizer step time, we also added total wall-clock with Unlearning only and End-to-end (ATI training and layers selection). Note that Figure 4's results now use the new tuned settings result, so the values have changed from 4-32x to 2-19x (considering the same per optimizer step time)

  • Weakness 2/Question 2: Unlearning objective ablation experiment was carried out that perform a single seed unlearning with each unlearning objective (as described in Section 3.2) ablated. We experiment with all 3 models and 2 datasets. Due to the large comparison results, we have included them in Appendix A.8

    To summarize, Full unlearning objective appear to be the best setting for 3/6 model-dataset, and ablating LqkL_{qk} to be the best for the other 3/6. However, ablating LqkL_{qk} only yields a maximum of 0.01 in Overall score,, but including LqkL_{qk} (as seen in full objective), yields a maximum of 0.05.

    Table 4 results have been updated to include new tuned results for our own method. We report ATI's results for first seed only, comparing with 4 other anchored selection. We added an NER tagger model as an additional baseline subject identification (https://huggingface.co/spacy/en_core_web_trf). Section 3.4.2 has been updated to reflect the new results.

  • Weakness 3/Question 3-4-5-6: TOFU setup uses forget10 as target unlearning data and retain90 as retain data (Description has been updated in Section 3.3 - Datasets) . We did not report the oracle reference TOFU model (fine-tuned on retain90 only) since the main goal is to benchmark our own method with other existing unlearning techniques

    "Variance and tuning": We added the 3 seeds run as well as tuning for both unlearn baselines and our own method. Table 2, 3, and 4 now include the new results, with standard deviation reported on Overall. To summarize, our method achieving best unlearning results in overall for 5/6 model-dataset, with the remaining TOFU Phi-4 being top 2 (0.04 behind UNDIAL).

    "Robustness": We have included a relearning attack influenced by https://doi.org/10.48550/arXiv.2406.13356](https://doi.org/10.48550/arXiv.2406.13356). The attack relies on benign samples that are relevant to the unlearn data without the explicit knowledge expression.

    For SUITE, we selected a subset of 100 samples each from Syntactic and Lexical in Retain Eval split. Syntactic includes samples that share the same question structure as forget data, while Lexical includes samples that share the same term (term is the unlearning subject in question) as forget data but different context.

    For TOFU, it is more difficult to find a dataset with the same forget authors set but different context. So we opted for the Real Authors dataset included in TOFU.

    The attack finetunes the selected dataset in 5 epochs with a fixed learning rate of 1e-05. We perform the attack on both our own method and baseline methods. Due to the large comparison, we have composed the results in a graph to be included in the Appendix A.7.

    To summarize, our method generally maintain unlearning effectiveness across models and datasets as reported by the 3 unlearning metrics, the only exception is Qwen2.5 on SUITE dataset where a slight increase to ES (0.55 --> 0.65) and EM (0.2-->0.4) are observed while ROUGE metric maintain low after 5 epochs. We also want to note that Qwen2.5 SUITE has only 3 layers targeted by our unlearning engine. It is possible that the increase in EM and ES are due to this smaller number of modified layers, leading to other retrieval pathway in Attention. However, such increases in EM and ES do not directly yield the actual leakage of sensitive information itself, as seen by ROUGE metric.

AuthorsOpenReview ↗

Summary of changes to the document:

  1. Table 2, 3 updated with new 3 seeds run and tuning results. λkv\lambda_{kv} descriptions in Table 2 and 3's captions are removed. Section 3.4.1 is modified to reflect the new results.
  2. Tables 9, 10, 11, 12 in Appendix A.6 are updated with the new seeded runs and tuning results.
  3. Table 4 updated with the new tuned result on first seed only. We also added comparison with anchored selected by spaCy NER.
  4. Figure 4 updated with total wall-clock and per-optimizer-step time. Reporting only relative speedup compared to mean baseline's speed. Relevant sections are also updated to reflect new results from Figure 4 (Section 3.5, Section 4, Section 1 - last paragraph).
  5. Update Equation 1 and Section 3.2 (first paragraph) to not explicitly determine a subject token selection threshold τ=0.75\tau=0.75
  6. Update Appendix A.5 to include detailed description and analysis of the tuning process as well as documented configurations.
  7. Add Appendix A.7 Relearning attack that shows details about the relearning attack.
  8. Add Appendix A.8 Unlearning objective ablation. Here, we ablate different components of out unlearn loss objective (Equation 2)
Reviewer_iZTTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes attention-only machine unlearning for LLMs. It introduces an Anchored Terms Identifier (ATI), trained on attention-head features from Mintaka, to identify entity-like tokens that act as lexical retrieval cues. The unlearning engine then updates only key/value projections in selected attention layers, while keeping query projections and other parameters frozen. Its objective combines attention suppression, representation steering toward “unknown” reference entities, projection preservation, and retained-data loss.

Critical strengths
  1. The proposed method achieves unlearning with a very small parameter footprint. Updating only the K/V projections of five layers (≤1.34% of parameters) gives the best composite score on all three SUITE models and on two of three TOFU models. This is against five established baselines, three modern 8–14B models and two benchmarks with different forget–retain overlap. On SUITE, which deliberately probes the anchor entities in benign retain contexts, the method has the highest retain ROUGE for every model.
  2. ATI is evaluated through both a causal probe and selector ablations. The results generally favor ATI-selected tokens over random or whole-input updates, supporting the paper’s argument that selective intervention can reduce collateral damage.
Critical weaknesses
  1. Core experimental details and statistical controls are missing. The paper does not define LretainL_{\text{retain}}, report random runs or uncertainty, or explain whether λ\lambda, the number of layers, ATI head budget, and threshold were selected on validation data. The custom harmonic-mean Overall score is central to the conclusions, but the metric transformations and tuning protocol are not sufficiently specified. These gaps make the comparisons difficult to reproduce and raise concerns about test-set selection or metric sensitivity.
  2. The unlearning evidence is limited relative to the claims. EM, extraction strength, output metrics, and quantization tests do not establish that the underlying information has been removed rather than obfuscated. The paper lacks benign relearning or stronger adversarial extraction evaluations. In addition, ATI is trained on Mintaka entity annotations and uses fictitious author-name references, so its transfer to real-world SUITE topics and other semantic domains is not fully demonstrated.
  3. The method only disrupts attention from anchor tokens and leaves the MLPs untouched. Knowledge may therefore remain reachable through queries that never name the anchor (indirect descriptions, coreference, paraphrases) or return after light relearning. The only attack tested is quantization, and INT4 already partly reverses forgetting on TOFU (EM up to +7 points). Without relearning or indirect-reference tests, the paper has not shown that the method removes knowledge rather than blocking one retrieval route.
Questions for authors
  1. How were λkvλ_{kv}, the learning rate and the five layers chosen, was any data held out from the reported forget and retain evaluation? Were the baselines tuned at all, for example their learning rate and number of epochs?
  2. Can the authors provide end-to-end wall-clock and optimizer-step comparisons including ATI training, feature caching, causal layer selection, and unlearning itself: how should the abstract’s 4–32× fewer optimizer step” claim be reconciled with the statement that all methods train for 10 epochs?
2 comments

We thank the reviewer for their insightful reviews.

We would like to address your concerns and questions as followed:

  • Weakness 1: Regarding LretainL_{retain} , we indeed did not include definition of this loss due to paper limit and only include losses definitions that are novel in our method. LretainL_{retain} is simply a cross-entropy loss on the retain dataset with coefficient λretain\lambda_{retain} control. ATI head budget is selected based on Table 1's result on Mintaka dataset, we opted for 10 head budget for downstream unlearning task.

    Tuning hyper-parameters λ\lambda, number of unlearning layers, and threshold were indeed selected mainly arbitrarily in the paper, along with a lack of tuning. We have since updated the document to include tuned unlearning results for both ours method and other unlearning baselines. The results also include 3-seed variants for unlearning (Table 2-3-4 updated). Due to paper limit, detailed description is updated in the Appendix A.5.

  • Weakness 2/Question 1: To further evaluate the knowledge removal of our unlearning technique, we added a relearning attack influenced by https://doi.org/10.48550/arXiv.2406.13356](https://doi.org/10.48550/arXiv.2406.13356) The attack relies on benign samples that are relevant to the unlearn data without the explicit knowledge expression.

    For SUITE, we selected a subset of 100 samples each from Syntactic and Lexical in Retain Eval split. Syntactic includes samples that share the same question structure as forget data, while Lexical includes samples that share the same term (term is the unlearning subject in question) as forget data but different context.

    For TOFU, it is more difficult to find a dataset with the same forget authors set but different context. So we opted for the Real Authors dataset included in TOFU. The attack finetunes the selected dataset in 5 epochs with a fixed learning rate of 1e-05. We perform the attack on both our own method and baseline methods. Due to the large comparison, we have composed the results in a graph to be included in the Appendix A.7

    To summarize, our method generally maintain unlearning effectiveness across models and datasets as reported by the 3 unlearning metrics, the only exception is Qwen2.5 on SUITE dataset where a slight increase to ES (0.55 --> 0.65) and EM (0.2-->0.4) are observed while ROUGE metric maintain low after 5 epochs. We also want to note that Qwen2.5 SUITE has only 3 layers targeted by our unlearning engine. It is possible that the increase in EM and ES are due to this smaller number of modified layers, leading to other retrieval pathway in Attention. However, such increases in EM and ES do not directly yield the actual leakage of sensitive information itself, as seen by ROUGE metric.

  • Weakness 3: As described in the paper, the SUITE dataset was created to evaluate exactly the different forms of input query that might result in the exposure of unlearned data.

    For example: A forget input -- Q: "What was the official mission code for the Space Shuttle Challenger's final flight?" A: "STS-51-L" Indirect form -- Q: "What was the mission code for the space shuttle that ended in disaster in January 1986?" A: "STS-51-L" Reverse form -- Q: "Which specific NASA space shuttle was lost during the mission officially coded as STS-51-L?" A: "Challenger"

    So our effectiveness in unlearning the SUITE dataset demonstrates the resistant to different input forms.

  • Question 2: After the new tuning process as mentioned above, we recalculated the training time and include both end-to-end wall-clock time and per-optimizer-step time (The amount of time it takes to complete one optimizer update). Our phrasing of "4–32× fewer optimizer steps" in the Introduction is misstated (as also mentioned by Reviewer LT8i), we have now changed it to be recent and accurate.

    In the new Figure 4 that compares our computational efficiency, apart from the new end-to-end wall-clock time, we also include the training time of our whole pipeline (ATI training, layers and head identification for unlearning) as well as RMU and NPO's training time on the same 5 layers target budget.

    Note that we did not include ATI training and layers identification in previous results due to the fact that they only need to be run once. RMU and NPO's same training budget of 5 layers is not an accurate statement as ours only target the Attention layers, while NPO and RMU targets 5 layers as a whole. So even with same 5 layers budget, the amount of parameters targeted is way less than RMU or NPO.

AuthorsOpenReview ↗

Summary of changes to the document

  1. Table 2, 3 updated with new 3 seeds run and tuning results. λkv\lambda_{kv} descriptions in Table 2 and 3's captions are removed. Section 3.4.1 is modified to reflect the new results.
  2. Tables 9, 10, 11, 12 in Appendix A.6 are updated with the new seeded runs and tuning results.
  3. Table 4 updated with the new tuned result on first seed only. We also added comparison with anchored selected by spaCy NER.
  4. Figure 4 updated with total wall-clock and per-optimizer-step time. Reporting only relative speedup compared to mean baseline's speed. Relevant sections are also updated to reflect new results from Figure 4 (Section 3.5, Section 4, Section 1 - last paragraph).
  5. Update Equation 1 and Section 3.2 (first paragraph) to not explicitly determine a subject token selection threshold τ=0.75\tau=0.75
  6. Update Appendix A.5 to include detailed description and analysis of the tuning process as well as documented configurations.
  7. Add Appendix A.7 Relearning attack that shows details about the relearning attack.
  8. Add Appendix A.8 Unlearning objective ablation. Here, we ablate different components of out unlearn loss objective (Equation 2)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.