acceptodds
Under review as a conference paper at ICLR 2027

Predicting the Collateral Cost of Linear Concept Erasure

Abstract

Linear concept erasure removes a target attribute from a frozen representation and guarantees linear guardedness: the optimal affine least-squares predictor of that attribute can no longer improve on the constant predictor. The guarantee says nothing about what the removal costs other attributes, and that cost is normally established only after the fact, by retraining a probe for every retained attribute. We show that the corresponding loss in linear accessibility can instead be computed in closed form beforehand. For any linear map that guards the concept set, the post-erasure accessibility matrix of the retained concepts is bounded above by a Schur complement of the accessible task covariance , whose diagonal entries are the linear of each concept and whose off-diagonal entries are the linearly accessible coupling between concept pairs, with equality for minimal-kernel erasers including LEACE. The Schur term is therefore the minimum collateral loss that guarding makes unavoidable, and for LEACE the leverage profile of the erased subspace additionally locates the edit. Across frozen representations and four datasets, both closed forms hold to within numerical error. Erasing every concept of every dataset in turn, the accessible coupling read off predicts held-out AUROC degradation of retrained probes better than squared label correlation does, and screening all CelebA triples from alone separates the cheapest erasures from the most damaging before any eraser is run.

Primary area:
interpretability and explainable AI
Keywords:
concept erasure, linear concept erasure, representation analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mvD9OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes an approach for quantifying the loss of information of a concept, when another concept is removed from the representation.

To provide some background: In deep learning (DL) we frequently rely on representations from large deep neural networks (DNNs) for various tasks (e.g. classification). We might want to remove a concept from such a representation (e.g. removing gender from the representations of CVs generated by a text encoder).

This paper seeks to quantify when you remove gender from this representation by achieving linear guardedness (e.g. no linear model can do better than a constant predictor to predict gender), how much information do you lose on other concepts (such as experience etc.). It quantifies this information via the R2R^2, and lower-bounds the decrease in R2R^2 for any projection that achieves linear guardedness (with an equality for the LEACE projection). The paper further argues that label correlation is insufficient to explain the loss in predictability of other concepts.

In the experimental section, it shows that this the terms used for the bound (the R2R^2) are more predictive of the ability to predict another concept (measured via AUC-ROC) than label correlation. They argue that this gives a computationally cheap way to screen beforehand which concepts to remove.

Critical strengths

The paper investigates a very important topic: namely, how much information is lost by the removal of a selected concept.

It has a solid approach to address this topic. Focusing on R2R^2 provides an analytical expression for quantifying information loss, which in turn reduces computation time. I also appreciated the investigation of LEACE, as it is a relevant method for this subfield.

The paper also provides an extensive empirical evaluation across a range of datasets and concepts, including numerical verification of the theoretical results. The experimental design is well motivated and appropriate for evaluating the proposed approach.

Critical weaknesses

I found this paper extremely hard to read and understand. Below, I will give some examples of this. My intention is not to be overly critical, but rather to offer constructive feedback that I hope will help the authors improve the presentation. The list below is not meant to be exhaustive but rather to illustrate a broader point.

  • The abstract contains many terms that are technical or specific to this literature that I would not expect all interested readers to know, such as ‘target attribute’, ‘probe’ ‘post-erasure accessibility matrix’, A0A_0 (I would in general avoid mathematical notation and terms in the abstract and introduction).
  • Similarly, for the introduction, there are many terms where I am still unsure what they mean. For example:
    • ‘a linear readout’ – while used in the literature, I encourage you to be more concrete for unfamiliar readers.
    • ‘second-order sense’ I do understand what you mean here but I do not think this is the accessible way of describing it.
    • ‘Linearly accessible means recoverable from the representation’ what does recoverable mean here?
    • ‘Some practical tasks’ What tasks?
  • In the methodology, I find the notation hard to follow: the notation does not distinguish between scalars, vectors, and matrices through capitalization and boldface. I would encourage the authors to adopt the conventions suggested by ICLR.
  • The paper frequently introduces theoretical results without explaining the concrete question they answer or intuition. For example, Theorem 1 could be introduced through this question: if we remove gender information from a representation, how much predictive power for occupation must necessarily be lost? This would help readers understand why the subsequent result matters. Similarly, I would encourage the authors to provide more intuition for Corollary 3 and Proposition 1, particularly their practical implications.

In terms of methodology, I believe the central weakness is that there is relatively little done to motivate that R2R^2 is a good metric for measuring information loss.It is not clear this is an good metric for (binary) classification tasks, where we typically rely on logistic regression. Indeed, Figure 1 suggests that while the decrease in R2R^2 is somewhat correlated with the decrease in AUROC, there is quite some dispersion, meaning that similar R2R^2 can correspond to different changes in the performance. I appreciate that the authors empirically investigate this limitation and acknowledge it. Nevertheless, I would encourage them to explain more clearly what their theoretical guarantees do and do not imply about downstream classification performance, and to better motivate why R2R^2 is the appropriate quantity to study for the intended applications.

Finally, I am not fully convinced of the usefulness of the proposed screening approach. The authors provide a runtime comparison in Section 5.3, showing that evaluating a candidate erasure takes 0.10 ms with their approach, compared to 3,339 seconds (less than an hour) for applying the erasure and evaluating downstream performance (for 12 concepts). This is a substantial speedup – but often we only care about a smaller number of concepts, and fitting a small number of linear probes is feasible. The advantage of fitting probes is that it provides a direct measurement of the loss, rather than a prediction based on R2R^2. I would encourage the authors to provide a more convincing practical example where the computational savings justify this trade-off.

Questions for authors

Why do all the experiments focus on frozen representations? I am not necessarily opposed to it, but it would be interesting to see if the experimental results also translate to the setting where we have fine-tuned the model. An experiment on a single dataset would already be informative.

4 comments

Dear Reviewer mvD9,

Thank you for the careful and detailed review. We especially appreciate your suggestions for improving the exposition and your questions about the connection between our theoretical metric and practical downstream performance. We agree that these distinctions should be communicated more clearly.

On presentation and intuition. We agree that the abstract and introduction introduce too much technical terminology before explaining the practical question. We will restructure the presentation around a practical example: if gender information is removed from a representation, how much linear predictive information about occupation must necessarily be lost? Theorem 1 answers this question exactly at the level of optimal linear least-squares accessibility.

Our goal is also to bring the main theoretical results on linear concept erasure together in one framework. This connects the geometry of the removed subspace, the maximum retained accessibility achievable by any guarding map, the conditions under which this optimum is attained, and the edit cost and footprint of LEACE. In particular, LEACE is shown to attain a universal optimum on retained accessibility, while the Schur term measures the collateral loss that no linear guarding map can avoid.

We will make this progression more intuitive, define concepts, probes, and linear accessibility earlier, clarify notation for scalars, vectors, and matrices, and introduce the theoretical results through the practical questions they answer.

On the choice of R2R^2 and downstream relevance. We agree that linear R2R^2 is not equivalent to AUROC or logistic-regression performance. It is nevertheless the natural theoretical quantity for our setting because linear guardedness itself is defined through the inability of an affine least-squares predictor to improve on a constant predictor. This makes accessibility loss exactly characterizable, whereas downstream classification degradation depends on the probe, regularisation, and data distribution.

Our empirical contribution tests whether this exact quantity is nevertheless informative about downstream performance. In the expanded single-concept evaluation, accessible coupling predicts held-out AUROC degradation better than squared label correlation (pooled Spearman 0.4590.459 versus 0.2120.212, two-way 95% CI for the difference [0.201,0.294][0.201,0.294]). We agree that this is a statistical association rather than a downstream performance guarantee, and will make that limitation more prominent.

On the practical value of screening. We agree that retraining a few probes may be entirely feasible when only a small number of candidate erasures are considered. Our intended application is different: screening many candidate concept sets before selecting which interventions to evaluate. The full Schur expression provides an exact pre-erasure score for linear accessibility loss, allowing candidate sets to be ranked without repeatedly applying erasers and retraining downstream probes.

To test whether screening works beyond the extreme candidates originally reported, we have now evaluated 30 additional pre-specified interior CelebA triples across ten representations, adding 300 actual erasures. On these non-extreme sets alone, the pre-erasure Schur score achieves Spearman ρ=0.912\rho=0.912 (95% two-way CI [0.76,0.96][0.76,0.96]) against held-out mean AUROC degradation, and ρ=0.909\rho=0.909 (CI [0.77,0.96][0.77,0.96]) against balanced-accuracy degradation. The measured AUROC drop increases monotonically across five predicted-score bins, from 0.0440.044 to 0.1080.108. This provides stronger evidence that the score is useful for candidate prioritization beyond the initially tested extremes. We do not claim calibrated downstream magnitudes or that screening replaces direct evaluation of selected interventions.

On frozen versus fine-tuned representations. Our focus on frozen representations is deliberate: the problem is to remove an attribute from an existing representation before training a downstream predictor. The theoretical results apply equally to a representation obtained from a fine-tuned encoder, provided that representation is subsequently held fixed. We agree that an explicit fine-tuned-encoder experiment would be informative, but it is not included in the present evaluation. We will clarify this scope and avoid implying that the reported empirical results establish transfer to that setting.

We appreciate your suggestions and will use them to make the paper's central message clearer: the framework characterizes the unavoidable collateral loss of linear guarding, identifies LEACE as attaining the optimal bound, and provides a practical way to screen candidate erasures before evaluating their downstream consequences. The accessibility guarantees are exact, while their relationship to downstream classification performance is empirical and necessarily more limited.

I thank the authors for their effort and novel manuscript. Below, I briefly go over the original concerns I raised and the extent to which I believe they have been resolved.

  1. I still find that the writing in the manuscript is overly technical, and hard to understand. Again, not to nitpick, but all the examples of unclear writing mentioned in the original review remain in the current version.
  2. I agree that R^2 is a legitimate proxy for downstream performance of logistic regression, and the additional results confirm this. As long as the authors are crystal clear about this limitation, I believe this is fine.
  3. I am still puzzled when you would want to 'screen many candidate concept sets before selecting which interventions to evaluate' - in most cases we have a small set of concepts (e.g. gender, race) that we definitinely want to remove. Could the authors provide an actual, real-world example of where one wants to screen this many concepts, and where it is an option to not remove some?
  4. The non-inclusion of finetuned representations is fine as long as it is mentioned as a limitation.

Overall, my strongest objection to the paper (quality of writing) remains, hence I will keep my score.

Dear Reviewer mvD9,

Thank you for revisiting our submission. We appreciate that our clarifications regarding R2R^2 and frozen representations have addressed two of your original concerns.

On presentation. We understand that the manuscript remains difficult to follow and acknowledge that our revision has not sufficiently addressed this concern. We appreciate your specific feedback and will use it to improve the exposition.

On screening. We agree that screening is less relevant when a fixed set of protected attributes must be removed. Our intended use case is optional shortcut removal, for example in medical imaging. Multi-site imaging representations can encode nuisance attributes such as hospital identity, scanner manufacturer, and acquisition year, which may contribute to shortcut learning (Zech et al., 2018; Hill et al., 2024).

Unlike mandatory protected attributes, these nuisance concepts need not all be erased. An auditor may consider multiple candidate subsets and seek interventions that preserve clinically relevant information. Our method ranks candidates by predicted collateral cost, allowing expensive erasure and downstream evaluation to focus on a shortlist. It does not determine which shortcuts should be removed or replace final validation.

We will clarify this distinction and strengthen the motivation in a future revision.

Thank you again for your constructive feedback.

I appreciate the authors getting back quickly. On screening: I believe you provide a great example of where screening might be useful. Would be a great addition to put this in there, accompanied by an experiment/dataset with multiple shortcuts (see for instance [1]).

I also appreciate the authors acknowledging the feedback on the writing and wish them good luck with a future revision. I would encourage them to continue working on this topic, I believe the general approach (quantifying linear information loss for many concepts) is definitely interesting.

[1] Li et al. 2022, https://arxiv.org/abs/2212.04825

Reviewer_GguTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper mainly focuses on the collateral cost in the task of linear concept erasure, while existing approaches only guarantee linear guardedness of the target attribute. The paper argues that this cost can be computed in closed form beforehand, not by retraining a probe for every retained attribute. Using the accessible task covariance A₀, where diagonal is the linear R^2 of each concept and off-diagonal is cross-concept coupling, retained accessibility is bounded above by a Schur complement of A₀. Therefore, the Schur term is the minimum unavoidable loss and is governed by accessible coupling rather than label correlation. Experimental results demonstrate that the proposed accessible coupling better predicts the collateral cost than label correlation.

Critical strengths

1. Clear and general theoretical result.

  • Theorem 1 is clear and general result that provides a Loewner-order upper bound on retained accessibility for any linear guarding map, not just LEACE, with equality held for the minimal guarding kernel. This theorem provides the explicit matrix-valued common value on top of the kernel invariance of Holstege et al. (2025).

2. Practical use as a pre-erasure screening tool.

  • This approach does not require any additional training or heavy computational cost, e.g., only 0.10 ms for scoring, and the experimental support that the predicted order agrees with the measured one at Spearman 0.794 is convincing.
Critical weaknesses

1. The theoretical contribution is incremental.

  • The key contribution of this paper is Theorem 1, which is mathematically correct, but it is quite incremental compared to the findings of Holstege et al. (2025). Theorem 1 only extends their shared value to the upper bound, and provides the matrix form of it. The remaining theoretical results are corollaries of this identity or restatements of known facts, so the theoretical contribution is limited.

2. The gap between theoretical and empirical results.

  • The pooled Spearman of 0.459 in Table 1 is the modest value, and the one for CelebA is not significant compared to the sole baseline (label correlation), which is the most interpretable benchmark. Moreover, there is no comparison against other baselines such as pre-erasure probe-logit correlation. The multi-concept predictor is a heuristic without theoretical grounding and loses to label correlation on the R² drop. The practical claims are thus not adequately supported by the experiments.
Questions for authors
  1. Can the authors provide the ranking accuracy and calibration in the middle of the range? The paper only provides two extreme cases like the top 5 and bottom 5 orders, so it is hard to validate the usage of the proposed measure.

  2. Do the authors already have the experimental results on other erasers, not LEACE? Theorem 1 covers any linear guarding map, but there are no experiments in the paper with linear concept erasers like INLP, SAL, and SPLINCE, not LEACE. Every experiment uses LEACE, whose equality case makes the bound an identity rather than a test of the inequality.

1 comment

Dear Reviewer GguT,

Thank you for the detailed and constructive review. We appreciate your recognition of the generality of Theorem 1 and the practical potential of pre-erasure screening. We address the concerns about theoretical novelty, empirical support, and the two specific experimental questions below.

On theoretical novelty. We agree that our results build on classical projection geometry and Holstege et al. (2025). However, our contribution goes beyond expressing their same-kernel invariance in matrix form. Theorem 1 establishes a universal Loewner-order bound over all linear guarding maps, showing that the Schur residual is the maximum retained accessibility compatible with guarding. LEACE attains this bound, making its collateral loss the minimum theoretically unavoidable cost rather than merely the outcome of a particular erasure procedure. Theorem 5 further characterizes exactly when other maps attain the same optimum, including maps with nonminimal kernels.

The paper also brings together previously separate geometric, statistical, and erasure-specific results into one framework connecting accessible task covariance, optimal retained accessibility, kernel invariance, and the LEACE edit cost and footprint. We agree that the underlying linear algebra is classical, but the resulting characterization of optimal guarding and its use for predicting collateral losses before erasure are our main contributions. We will make this distinction clearer.

On the empirical evidence. We agree that pooled Spearman ρ=0.459\rho=0.459 is moderate and that the CelebA advantage is not statistically resolved. However, the submission already includes an expanded evaluation across 122 erased concepts, 40 representations, 1,220 erasures, and 42,280 retained-concept observations. Table 1 reports ρ=0.459\rho=0.459 versus 0.2120.212 for label correlation, with Δρ=+0.247\Delta\rho=+0.247 and a two-way clustered 95% CI of [0.201,0.294][0.201,0.294]. The advantage also persists across logistic regression with C∈{0.1,1,10}C\in\{0.1,1,10\} and linear SVM (Appendix E.5), with margins between +0.230+0.230 and +0.242+0.242.

On the multi-concept heuristic and R2R^2 comparison. We agree that the maximum pairwise coupling used in Appendix E.3 is heuristic. However, this should be distinguished from the candidate-set screening in Section 5.3, which uses the full theoretically exact Schur expression ATSASS†ASTA_{TS}A_{SS}^{\dagger}A_{ST} for arbitrary erased sets.

The reported R2R^2 reversal also concerns a specific heuristic set-level comparison. Normalized coupling predicts relative R2R^2 loss, whereas that comparison uses absolute loss. At matched scaling, the relationship is an algebraic identity (MAE 1.7×10−131.7\times10^{-13}). Even against absolute R2R^2 loss, the expanded single-concept analysis yields pooled ρ=0.908\rho=0.908 versus 0.7210.721 for label correlation (Section 5.2, Table 19). We will clarify these distinctions. We also agree that additional baselines, including pre-erasure probe-logit correlation, would strengthen the comparison.

Q1. Ranking accuracy through the middle of the range. Thank you for this suggestion. We conducted a new experiment with 30 pre-specified interior triples, selected deterministically across previously unmeasured ranks 6-450 before observing any new outcomes. Each was evaluated with actual LEACE erasure and held-out logistic probes on the same ten CelebA representations, adding 300 erasures.

On these 30 non-extreme sets alone, Spearman correlation between the pre-erasure Schur screening score and measured mean retained AUROC degradation is ρ=0.912\rho=0.912 (95% two-way bootstrap CI [0.76,0.96][0.76,0.96]; Kendall τ=0.752\tau=0.752). For balanced accuracy, ρ=0.909\rho=0.909 (CI [0.77,0.96][0.77,0.96]; τ=0.743\tau=0.743).

Across five consecutive predicted-score bins, observed mean AUROC degradation increases monotonically:

0.044→0.073→0.076→0.091→0.108.0.044\to0.073\to0.076\to0.091\to0.108.

All 40 evaluated triples combined yield ρ=0.957\rho=0.957 for AUROC and 0.9560.956 for balanced accuracy. We emphasize the 30 new interior sets as the primary evidence so that the originally selected extremes do not drive the result. These findings support ranking and prioritization, not calibrated prediction of downstream magnitudes. We will add the experiment, pre-specified selection protocol, and figure to the appendix.

Q2. Other erasers. We agree that the experiments currently validate the equality case using LEACE rather than the strict inequality for maps with larger effective kernels. The theorem covers arbitrary linear guarding maps, but empirical verification of that broader case remains an appropriate extension. We will clarify this limitation and distinguish same-kernel alternative erasers, which also attain equality, from maps that remove additional directions.

Thank you again for these questions, particularly the middle-range screening suggestion, which motivated a substantial additional validation.

Reviewer_NvzuOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the collateral cost of linear concept erasure on retained concepts. The authors introduce an accessible task covariance matrix that characterizes the linear predictability of individual concepts and their coupling. Their main theoretical result establishes an upper bound on the retained linear accessibility after erasing sensitive concepts, expressed through a Schur complement. They show that LEACE attains this bound and further characterize its edit cost and footprint through rank and leverage scores. The paper also presents empirical evaluations across multiple datasets and pretrained representations.

Critical strengths
  • The paper provides a clear mathematical formulation of collateral damage in terms of linear predictability.
  • The theoretical results are accompanied by detailed proofs and extensive empirical evaluations.
Critical weaknesses

My main concern is the limited theoretical novelty of the proposed results. The central characterization appears to be a rather direct application of established results from classical linear algebra and multivariate statistics, particularly canonical correlation analysis (CCA), least-squares regression. The subspace removed by LEACE coincides with the canonical subspace identified by CCA in whitened coordinates. Consequently, the characterization of retained linear predictability after removing this subspace follows naturally from standard projection and residual covariance identities. Similarly, the Schur-complement expression and its upper-bound interpretation are well-established concepts in matrix analysis (e.g., Anderson, 1971). The connection to existing concept-erasure results, including Belrose et al. (2023) and Holstege et al. (2025), further suggests that much of the underlying geometry is already understood. While the paper applies these mathematical tools to characterize collateral damage in concept erasure, I find it difficult to identify a substantial new theoretical insight beyond the application and reformulation of these established results. In particular, it remains unclear which aspects of the main theorem go beyond standard consequences of the existing projection framework. Given that the paper's primary contribution is theoretical, I consider this a significant limitation. The authors should clarify what genuinely new mathematical insight their analysis provides beyond these established results and why the resulting characterization constitutes a substantial theoretical advancement.

Questions for authors

please refer to the weakness section.

2 comments

Dear Reviewer Nvzu,

Thank you for your careful review and for highlighting the question of theoretical novelty. We agree that the mathematical tools underlying our analysis, including orthogonal projections, least-squares regression, and Schur complements, are classical. Our contribution is not a new matrix identity, but a unified characterization of collateral loss under linear concept erasure, including an optimality result over the full class of linear guarding maps.

On the distinction from Holstege et al. (2025). Their same-kernel invariance result establishes that erasers sharing a kernel preserve identical predictions under unregularised refitting. Our Theorem 1 addresses a broader optimization question: among all linear maps that successfully guard a target concept set, what is the maximum retained linear accessibility achievable? We prove that every such map satisfies

A0(PZ,YT)⪯ATT−ATSASS†AST.A_0(PZ,Y_T)\preceq A_{TT}-A_{TS}A_{SS}^{\dagger}A_{ST}.

with equality attained by LEACE. Thus, LEACE is not merely one eraser whose collateral loss can be calculated. It attains the universal upper bound on retained accessibility, while the Schur term gives the minimum unavoidable collateral loss over the entire feasible class of guarding maps. This optimality interpretation does not follow from same-kernel invariance alone.

On the structural content beyond the Schur identity. Theorem 5 (Appendix A.8) further establishes that a minimal guarding kernel is sufficient but not necessary for equality. An eraser may remove additional directions without incurring additional collateral loss precisely when its surplus-erased subspace is orthogonal to the surviving retained accessible subspace. This characterizes when the optimal bound remains attainable even beyond minimal-kernel erasers.

On the broader theoretical contribution. We also view the unification of previously separate results as an important part of the work. The framework connects accessible task covariance, the geometry of the LEACE projection, same-kernel invariance, the universal collateral-loss bound, its exact equality conditions, and the LEACE-specific edit cost and leverage footprint. Together, these results explain what linear guarding must remove, how much information can be preserved, and how LEACE performs the edit. The resulting quantities can be computed from pre-erasure statistics without executing the intervention.

We agree that the individual mathematical methods and theory are established. Our claim is that bringing them together gives a new characterization of optimal linear concept erasure and a way to predict unavoidable collateral losses before erasure. We will emphasize this distinction more clearly and position the contribution accordingly.

We appreciate your feedback, which helps us clarify both the scope and significance of the theoretical results.

Thank you for the clarification. I accept that Theorem 1 goes beyond the same-kernel invariance result of Holstege et al. (2025). However, I still think my main concern about the theoretical novelty remains.

From standard least-squares projection theory, we know that for any linear map (P), we have 0⪯AP⪯A00 \preceq A^P \preceq A_0. Since guarding the sensitive concepts makes the corresponding blocks of APA^P zero, the proposed upper bound seems to follow directly from applying the standard Schur-complement inequality to A0−AP⪰0A_0-A^P \succeq 0. Similarly, the equality for LEACE seems to follow from its established projection geometry.

I think the theoretical results are useful, but the contribution still seems incremental to me, since most of it follows from established results. I would have liked to see a more convincing practical application that shows what this theory offers other than what we already know from existing methods.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.