- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ERASER, a representation-guided method for LLM unlearning. It constructs a fixed forgetting subspace from frozen-model activations and penalizes the components of forget representations that lie in this subspace, while a ranking-based Disentangle-Head encourages retain samples to receive higher scores than forget samples. Experiments on WMDP, TOFU, MUSE, and Fictional Knowledge report competitive forgetting–retention trade-offs, along with additional evaluations of jailbreak attacks, membership inference, and limited-data relearning.
Critical strengths
-
ERASER replaces RMU’s random target with a data-derived forgetting subspace, providing a more interpretable representation-level objective without requiring a manually selected target scale.
-
The evaluation is broad, covering four unlearning benchmarks, multiple model families, component ablations, abstraction-level probes, jailbreak attacks, membership inference, and limited-data relearning.
Critical weaknesses
i) ERASER constructs its forgetting subspace from the leading directions of uncentered forget-set activations, without contrasting them with retain-set activations. These directions may therefore capture common high-variance language features rather than knowledge specific to the forget set. Unlike LEACE, which defines concept erasure through the inability of linear classifiers to recover the target concept while minimizing changes to the remaining representation, ERASER does not provide an operational test showing that its projected directions encode the knowledge to be removed. Prior work has also shown that projection-based removal may unintentionally affect information unrelated to the target [a]. This concern is reflected in Appendix B.6, where unlearning Economics and Law reduces accuracy on the adjacent Econometrics and Jurisprudence tasks by 13.16 and 23.15 percentage points, respectively.
The ablation results further suggest that most of the improvement comes from the Disentangle-Head rather than the proposed projector. In Table 12, adding the same head to RMU already produces results close to the full ERASER method: ERASER improves Bio and Cyber accuracy by only 0.71 and 1.31 percentage points over RMU with the head, while slightly reducing MMLU. Moreover, the ranking head can learn to separate forget and retain samples through its own parameters without making the underlying representation space geometrically disentangled. A stronger comparison against RMU with the same head, together with direct measurements of target-information recoverability and forget-retain subspace overlap, is needed to establish the contribution of subspace elimination.
ii) The membership-inference experiment compares forget samples with controls drawn from different corpora, so it cannot separate membership information from differences in data source or content. The paper acknowledges this limitation, but the resulting scores provide little evidence about privacy after unlearning. The jailbreak results are also difficult to interpret because the qualitative examples show that ERASER often produces fragmented or malformed responses. Its lower GCG attack success rate may therefore reflect degraded output quality rather than removal of the target knowledge. This distinction matters because output-level suppression can coexist with recoverable information in the model’s internal representations [c]. Under the embedding-space attack, the cumulative attack success rate still reaches 78%, further limiting the robustness claim.
The relearning evaluation is similarly incomplete. It fine-tunes only ERASER on 10, 30, or 50 examples from the original forget set and does not compare its recovery with RMU or other unlearning methods. It also omits retain-data and unrelated-data controls. Fine-tuning on retain data would show whether ordinary continued training can reactivate the target knowledge, while unrelated data would help determine whether any recovery reflects residual knowledge or merely a general effect of additional fine-tuning. Prior work has shown that apparently forgotten knowledge can reappear after benign relearning [b], making these controls and baseline comparisons important before claiming stronger resistance to recovery.
[a] Better Hit the Nail on the Head than Beat around the Bush: Removing Protected Attributes with a Single Projection. EMNLP 2022.
[b] Unlearning or Obfuscating? Jogging the Memory of Unlearned LLMs via Benign Relearning. ICLR 2025.
[c] An Illusion of Unlearning? Assessing Machine Unlearning Through Internal Representations. AISTATS 2026.
Questions for authors
-
Can the authors directly test whether the projected subspace contains forget-specific knowledge rather than common high-variance features, for example through linear probes, forget-retain subspace overlap, or comparison with a retain-conditioned concept-erasure method?
-
In Table 12, RMU with the Disentangle-Head performs very similarly to the full ERASER method. Are the remaining differences statistically significant across random seeds, and what evidence shows that the subspace projector contributes meaningfully beyond the ranking loss?
-
How does ERASER compare with RMU and other baselines under the same relearning protocol? Can the authors also include retain-data and unrelated-data controls to determine whether recovery is specific to residual target knowledge or a general consequence of additional fine-tuning?