acceptodds
Withdrawn from ICLR 2027

Neuralyzing the Trace: Selective Representation-Level Unlearning with Contrastive Sparse Autoencoders

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Machine unlearning aims to remove targeted information while preserving a model's other abilities. In realistic settings, such as privacy requests under the EU GDPR, the target may be narrow, for example information associated with a single person. Behavioral forgetting alone may be insufficient, motivating interventions directly on internal representations. However, standard mechanistic-interpretability extractors are poorly selective for such targets. We identify an energy bias in reconstruction-based extraction, which favors dominant background structure over low-energy target-specific components. We introduce SCALPEL, a contrastive sparse autoencoder designed to learn more selective forget features. We show theoretically that contrastive training promotes target-selective features and that our selection score controls expected background knowledge perturbation. We validate SCALPEL experimentally on TOFU across Qwen, Llama, and Gemma, where it substantially improves over NMF and standard SAE interventions and is competitive with Gradient Difference and RMU, bridging mechanistic interpretability and fine-grained unlearning.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Contrastive Learning, Machine Unlearning, Concept-based Interpretability, Large Language Models, Sparse Autoencoders, Representation Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_p5Q5OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the energy bias of reconstruction-based feature extractors, such as sparse autoencoders (SAEs), which can prioritize dominant background structure over low-energy target-specific components. To address this limitation, the authors propose SCALPEL, a contrastive SAE framework for fine-grained representation-level unlearning. SCALPEL learns target-selective sparse features by contrasting multiple views of each forget target against retain and Wiki examples. At inference time, selected feature contributions are subtracted from residual-stream activations in an input-dependent manner, without modifying the original model parameters.

Critical strengths

The paper identifies an important limitation of reconstruction-based feature extraction for fine-grained unlearning and provides a theoretical explanation of the energy bias induced by reconstruction objectives.

Critical weaknesses

Insufficient empirical evidence connecting the energy gap to forget-specific information: In Section 3 and Figure 2, the authors report an approximately 329-fold energy gap between the top-three PCA components of background activations and the top-three PCA components of forget activations after removing the dominant background directions. However, these quantities are computed from different distributions and under different transformations. Moreover, the background-orthogonal residual is not necessarily equivalent to forget-specific information, as it may still contain other shared or irrelevant variation. Although Appendix A.3 clarifies that the reported gap does not represent the energy ratio of the complete forget and background manifolds, the evidence remains insufficient to establish that the measured residual energy specifically reflects forget-target information or that the observed gap explains the selectivity limitations of reconstruction-based extractors. The authors also state that the observed gap is substantially larger than that obtained under a random split, but the corresponding numerical results and distributions are not provided. This claim could be strengthened through: (1) quantitative distributions of energy gaps under matched random or pseudo-forget splits using the same analysis pipeline; (2) an intervention-based analysis verifying whether the residual components actually encode forget-specific information; and (3) an analysis relating target-specific energy gaps to the unlearning selectivity of reconstruction-based methods and SCALPEL.

Lack of a Tabulated Comparison at Matched Forgetting Levels: Although the paper presents forgetting–utility trade-off curves in Figure 3 and Appendix F, it does not provide a consolidated numerical comparison of the methods at matched forgetting levels. Such a comparison would make it easier to assess the practical performance differences between SCALPEL and the key baselines, particularly RMU and Gradient Difference. I recommend reporting quantitative results at matched forget-set ground-truth answer probabilities, together with corresponding retain-set utility and Wiki performance. Relevant forgetting metrics, such as ROUGE, truth ratio, and privacy leakage, could also be included at the selected operating points. In addition, evaluating general reasoning capabilities would help determine whether the intervention preserves broader model utility beyond the existing TOFU and Wiki evaluations.

Questions for authors

SCALPEL uses retain and Wiki activations as background examples for contrastive learning and feature selection. How sensitive are the selected features and unlearning performance to the choice of background distribution? In particular, would the method remain effective when replacing Wiki with another dataset or using only the retain set?

Section 4.2 motivates the use of the fourth-to-last layer as a compromise between factual representation and next-token prediction. However, how sensitive is SCALPEL to the intervention layer? Does the same choice consistently provide strong target selectivity and unlearning performance across Qwen, Llama, and Gemma?

Section 4.4 describes the roles of common/difference losses, but their precise formulations and contrastive pair construction remain unclear. Could the authors explicitly define both objectives, including their positive and negative sampling strategies and how the two latent blocks are trained?

Reviewer_n2mCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the concern that existing unlearning methods often suppress target information only at the behavioral level, without removing the underlying knowledge. This study proposes SCALPEL, a method that intervenes directly on internal representations with improved selectivity. The authors first argue that reconstruction-based extractors allocate capacity preferentially to high-energy background structure, which makes it difficult to selectively isolate low-energy, target-specific components of the forget set. They support this with an energy analysis of Qwen’s residual stream using PCA: the top-3 background energy is about 329× larger than that of the forget residual after the dominant background directions are removed. Building on this, SCALPEL generates multiple augmented views of each forget author and adds a contrastive loss to SAE training. Views of the same author serve as positives, and retain/Wiki tokens serve as negatives. The latent code is further split into a common block and a target-specific block. After training, features from the target-specific block are selected by a selectivity score. At inference, their decoded contributions are subtracted from the original activation, scaled by α. Experiments on TOFU with Qwen2.5-1.5B, Llama-3.2-1B, and Gemma-2-2B show substantially better forget–utility trade-offs than NMF and standard SAE interventions, and performance competitive with RMU and Gradient Difference.

Critical strengths
  • The paper clearly recognizes that suppressing output behavior differs from actually removing knowledge (“behavioral evaluation alone cannot establish that knowledge is gone”), and from this perspective it convincingly motivates intervening directly on internal representations. It also pinpoints why existing MI-based extractors fail on narrow targets: a lack of selectivity.
  • The related work on LLM unlearning is well organized. It covers weight-based, inference-time, model-editing, and MI-based approaches, and it clearly positions the proposed method relative to them.
Critical weaknesses
  • The experiments are limited to TOFU and small models. TOFU injects fictitious author information through fine-tuning, so the resulting knowledge may be more localized and easier to separate than real knowledge that is broadly entangled during pretraining. Moreover, the method’s target definition (author-level grouping) relies heavily on TOFU’s structure. The paper presents realistic requests such as GDPR as its core motivation, yet it defers evaluation on MUSE and WMDP to future work. The authors’ own theory predicts that the advantage may shrink on datasets where targets carry higher energy, which makes validation on other benchmarks even more necessary. The baselines also need strengthening: Gradient Difference is a relatively weak baseline, and NPO-family methods, although discussed in the related work, are not included in the comparison. In addition, several important design and validation questions are repeatedly deferred to future work throughout the paper, including layer selection, MUSE evaluation, sequential unlearning, and target granularity. This reduces the completeness of the current results.
  • The paper is motivated by “removing knowledge rather than suppressing behavior.” However, SCALPEL is an inference-time filter that leaves the weights unchanged, so the original information remains accessible once the intervention is removed. The authors acknowledge this as a limitation. Given this, experiments verifying that the target information is absent at the representation level seem necessary at minimum, for example probing for the target information at layers after the intervention point, or testing robustness to relearning. The current evaluation relies mainly on output-based metrics, which leaves a gap between the motivation and the validation.
Questions for authors
  • What α values were finally selected for each model? α = 1 removes exactly the feature’s contribution, while α > 1 pushes the activation further in the opposite direction. Since the search grid extends up to 32, if the optimal α is substantially greater than 1, can this still be regarded as “feature removal”? It would also help to know what form the model’s outputs take in that regime (plausible wrong answers, refusals, or degenerate text).
  • How are concept identifiers assigned to retain/Wiki tokens? Is each retain author given a separate identifier, or are they grouped into one? Also, are the token positions I(x) used for activation extraction restricted to answer tokens, or do they cover the full sequence?
  • What is the exact energy gap under a random split? The paper states only that the 329× gap is “far larger” than under a random split, but the concrete number is needed to judge how strongly this measurement supports the energy-bias argument.
  • Could you report the wall-clock cost of the full pipeline per deletion request (view generation, activation caching, SAE training) and the inference-time overhead, compared with RMU and Gradient Difference?
Reviewer_oUKDOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper targets fine-grained LLM unlearning (one TOFU author) through inference-time intervention on residual-stream features, arguing that standard SAE/NMF/PCA extractors are not selective for narrow targets because reconstruction objectives favor high-energy background structure. It proposes SCALPEL, a sparse autoencoder trained with a multi-positive InfoNCE loss, selects features by a target-to-background activation ratio, and subtracts their decoded contributions from the activation at inference without changing weights.

Critical strengths

The hyperparameter search and multi-seed protocol are fully documented (Appendix E: search spaces, budgets, pruning, selection objective, post-hoc overrides), which makes the results reproducible even though the baselines and metrics chosen are inadequate.

Critical weaknesses
  1. The baselines are incomplete. GradDiff and RMU are the only weight-based methods, while NPO, SimNPO and IdkDPO are standard on TOFU and already available in the OpenUnlearning codebase the authors use. There is also no comparison to any existing SAE-based unlearning method (DSG, conditional clamping, CRISP, ARIA). On top of that, RMU and GradDiff are shown as single points against full SCALPEL curves, and even so RMU is better than SCALPEL on Qwen and Llama in Figs. 3 and 7. Based on these results, the proposed method doesn't seem competitive against an incomplete list of existing baselines.

  2. The main ideas have been published before and the contribution of this paper seems rather incremental. Ratio-based feature selection with input-conditional clamping is DSG (arXiv 2504.08192) and Khoriaty et al. (2503.11127); weight-free SAE unlearning on TOFU is ARIA (2609.16229); label-conditioned or contrastive SAEs used to suppress a concept appear in SCAR (2411.07122) and Kim et al. (2605.12122); and the observation that SAEs trained on broad data spend their capacity on frequent patterns and miss rare concepts is in Muhamed et al. (2411.00743) and O'Neill et al. (2508.09363). The paper's "energy" framing is a rewording of that known problem.

  3. The writing is not very polished and the paper is generally not easy to follow (e.g. inconsistent method naming across Sec. 4.4, Table 3, and Fig. 3; empty Appendix F headings; Fig. 6 duplicating Fig. 3).

Questions for authors

None

Comments on the paper

1 comment

Thank you for the time you spent reviewing our paper. Your feedback was very useful. We have decided to withdraw our paper because, after conducting more advanced experiments, we realized that it still needs more work and maturity.

Thank you again for your feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.