acceptodds
Withdrawn from ICLR 2027

Absorption Aware Editing: Predicting and Closing the Leakage Channels of SAE Based Concept Erasure

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Sparse autoencoders (SAEs) provide interpretable latent features that can be used to edit concepts in language models without retraining. The usefulness of such edits depends on concept-associated latents providing reliable control over the corresponding information in the model representation. However, ablating these latents often leaves substantial concept information behind, and existing methods provide little guidance about when these edits will fail or how to repair them. We show that the extent of feature absorption, a known SAE pathology, quantitatively predicts the resulting concept leakage, and surviving information is distributed across absorbing SAE latents and the reconstruction residual. Based on this decomposition, we propose a training-free, absorption-aware editing method that removes the target concept direction from both. Across multiple SAE training methods and model families, the proposed method substantially reduces concept leakage with minimal effects on unrelated concepts and model utility. Together, these results turn a systematic source of SAE editing failure into one that can be diagnosed, predicted, and corrected.

Primary area:
interpretability and explainable AI
Keywords:
Sparse Autoencoder, Concept Erasure

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dujmOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies why SAE-based concept erasure can leave target information behind after the identified concept latents are removed. It shows that feature absorption predicts this leakage, as the concept may instead be represented by other SAE latents or the reconstruction residual. Based on this observation, the paper proposes a training-free operator that removes target-aligned components from both channels. Experiments across several SAE variants, Gemma models, and orthographic, syntactic, and semantic concepts show substantially lower probe-based leakage with limited changes to non-target information and general model utility.

Critical strengths
  • This paper identifies feature absorption as a concrete and measurable source of failure in SAE-based concept erasure, supported by consistent token-level and cross-SAE results.

  • The proposed operator is simple and training-free, and the experiments cover multiple SAE training methods, three model scales, and several concept types while showing limited effects on non-target information and model utility.

Critical weaknesses

i) The central relationship between feature absorption and editing failure is partly built into the definitions and evaluation protocol.

A fully absorbed example is required to have inactive parent latents while remaining detectable by the original concept probe. Since standard parent ablation removes only the parent contributions, it leaves the activation of such an example unchanged. Therefore, the near-perfect leakage rate on fully absorbed examples follows directly from their construction, as the paper itself notes in Appendix A.3. Similarly, the absorption fraction and post-edit margin are both calculated using the same probe direction. The strong correlation in Figure 1 therefore does not provide fully independent evidence that absorption predicts editing failure.

The cross-SAE analysis has a related issue. Overall leakage is calculated directly from the proportion of fully absorbed examples and their leakage rate, which is already close to one by construction. Using the proportion of fully absorbed examples to predict overall leakage in Table 9 consequently introduces a direct mathematical dependence between the predictor and the target. The authors should separate this definitional relationship from genuine out-of-sample predictive value by using independently trained probes, holding out entire concept families rather than only SAEs, and adopting leakage measures that are not constructed from the same absorption statistics.

ii) The experiments do not isolate the contribution of the absorption-aware design or independently establish that the target concept has been erased.

The method uses the same fixed probe direction to define absorption, select candidate latents, perform the edit, measure leakage, and construct the recovery attack. Because the operator explicitly removes information along this direction, a low score from the same probe is an expected consequence of the intervention and does not show that the concept is no longer linearly or nonlinearly recoverable. Existing concept-erasure methods instead retrain classifiers after each projection [a], optimize against an adaptive classifier [b], or guarantee that no linear classifier can recover the target concept [c]. Prior probing work also cautions that probe performance alone does not establish how a representation affects model behavior [d]. The paper should therefore evaluate newly trained linear and nonlinear probes and include concept-related generation or downstream behavioral tasks.

The paper also omits a direct baseline that projects the entire edited activation away from the target probe direction. Without this comparison, it is unclear whether the improvement comes from the proposed decomposition across absorbing latents and the reconstruction residual or simply from removing the evaluator’s concept direction. Moreover, the candidate set includes every non-parent latent whose decoder direction exceeds a cosine threshold, rather than only latents independently identified as absorbing the target concept. With the threshold used for Gemma Scope 2, Table 16 shows that approximately 33%–39% of all latents are selected. The threshold also changes across SAE families without a general calibration rule. These results make the operator look more like broad probe-guided projection than a selective correction of absorption channels.

[a] Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection. ACL 2020.

[b] Linear Adversarial Concept Erasure. ICML 2022.

[c] LEACE: Perfect Linear Concept Erasure in Closed Form. NeurIPS 2023.

[d] Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals. TACL 2021.

Questions for authors
  1. How does the proposed method perform when concept leakage is evaluated using newly trained linear and nonlinear probes rather than the same fixed probe used to construct the edit?

  2. How does the absorption-aware operator compare with a direct baseline that projects the entire edited activation away from the target probe direction, using both leakage and collateral-damage metrics?

  3. How should the cosine threshold be selected for a new model or SAE without using evaluation leakage, and does the method remain selective when the candidate set contains more than one-third of all SAE latents?

Reviewer_SbZxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

SAEs can suffer from feature absorption: when a parent latent is inactive, a child latent carries the parent’s information. This may explain why editing fails. This paper proposes a training-free, absorption-aware editing method to better remove target concepts. Experiments on language models (SAEBench setting) and across different SAE architectures show that the introduced operator improves erasure.

Critical strengths
  • The paper shows that parent ablation is the wrong edit for much of the signal, since the SAE residual and non-parent latents carry it
  • The proposed method is training-free and can be applied to different SAEs architectures
Critical weaknesses
  • All experiments use Gemma models and Gemma-family SAEs, so the general claims about SAE-based erasure are tested on a narrow slice. Results on a non-Gemma model (and, as a stretch, a non-language SAE such as a diffusion SAE) would show whether the absorption–leakage relation is general.
  • See the questions below for additional concerns that could improve the paper.
Questions for authors
  • After editing, what is a retrained probe’s accuracy on the edited activations?
  • If the candidate set is limited to children defined independently of the probe cosine (for example, by decoder similarity to, or co-activation with, the parent), or to the single top-contributing latent j*, how much of the leakage reduction remains?
  • What fraction of the removed signal comes from true children of the parent vs latents that pass τ by chance?
  • Why is the residual projection applied with its sign, while negatively aligned latents are deliberately excluded from C?
  • As you grow the edited set (active non-parent latents), how do leak_A and leak_N change?
  • Can this method be applied to SAEs trained on diffusion models? Such as SAEmnesia or SAeUron checkpoints on UnlearnCanvas. If the authors can conduct preliminary experiments in non-language-only settings, I would view evidence that the method generalizes beyond Gemma language models positively, and it could strengthen my assessment.
Reviewer_uUtSOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies a concrete failure mode inherent to SAE-based editing and puts forward a simple solution grounded in solid motivation. Extensive experiments across different SAE architectures and Gemma model scales show effective leakage reduction (Tables 1–6). The major concern lies in the reuse of one probe direction for both designing the intervention and evaluating concept removal; this reduces the strength of evidence that the target concept has been truly erased. The manuscript is generally coherently organized, while some figures and formatting elements could be refined.

Critical strengths

1.The paper provides a clear motivation by identifying feature absorption as a likely source of failure for SAE interventions. Its editing operator is directly derived from the representation decomposition in Eqs. 2–3 and needs no supplementary model training. 2.The approach reliably cuts probe leakage versus the parent ablation and other baselines (Tables 1–6). Ablations verify the importance of non-parent latents and reconstruction residuals. Tests on multiple SAE architectures and Gemma scales demonstrate acceptable robustness (Tables 1–4). 3.The paper is generally well organized, with a clear structure and readable experimental tables. In particular, Fig. 2 provides an intuitive visualization of the proposed editing mechanism and helps clarify the overall methodology.

Critical weaknesses

1.The same probe direction is reused for concept localization, intervention editing, and leakage assessment. As a result, reduced leakage measured by this fixed probe does not conclusively demonstrate true erasure of the target concept. Evaluations using independently trained linear or nonlinear probes would substantially reinforce the paper’s central claim. 2.Nearly all experimental outcomes are reported as point estimates without accompanying confidence intervals. Furthermore, all language models tested are from the Gemma family. These choices weaken both the statistical reliability and cross-model generalizability of the findings. 3.The threshold varies across SAE setups with insufficient explanation of its selection rule. Figure 1 also suffers from cramped, overlapping labels.

Questions for authors

1.The proposed intervention explicitly eliminates components aligned with the probe direction w^L\hat w_L, yet leakage is quantified using this identical fixed probe. Could the authors evaluate whether the purportedly erased concept can still be recovered by retraining a separate linear or nonlinear probe on the edited representations? 2.Could you report a direct whole-activation projection baseline, [ x' = x-\langle x,\hat w_L\rangle \hat w_L,] using the same evaluation protocol? This would help disentangle how much performance gain arises specifically from the absorption-aware decomposition, instead of merely suppressing the probe direction. 3.How is the threshold τ\tau selected across different SAE configurations? Is it picked based on a held-out validation split, or tuned directly on test data?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.