acceptodds
Under review as a conference paper at ICLR 2027

Operational Abstractions of Neural Network Concepts via Topological Representations

Abstract

Concept-based methods provide a semantic level for interpreting and manipulating learned representations, but existing editing approaches are typically specialized to particular interventions and do not provide a common and editable representation of concept organization. To achieve this, we introduce Topological Concept Representations (TCR), a post-hoc operational abstraction that jointly characterizes the concepts encoded in a learned representation and their relationships. TCR constructs an intermediate concept space from concept recoverability and interaction scores, and compactly encodes its organization through topology. Interventions are expressed through modifications of this abstraction that are propagated back to the underlying learned representation. This allows different concept-level operations to share the same optimization framework and separates the desired concept organization from the mechanism to achieve it. We establish stability and reparameterization-invariance properties of TCR and its connections to existing concept-editing formulations. We use TCR to disentangle concepts as a preprocessing step for existing erasure methods, improving worst-group accuracy by 21.89 on average at comparable concept leakage. We further use TCR to transfer concepts from teacher to student models, improving concept recoverability by up to 5.54 while also improving or maintaining competitive test top-1 accuracy.

Primary area:
interpretability and explainable AI
Keywords:
Topological Data Analysis, Concept-based Interpretability, Concept Erasure, Concept Disentanglement, Concept Transfer

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_HqRYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Topological Concept Representations (TCR), an abstraction that jointly represents concepts and their relationships and provides a common interface for concept-level interventions. It provides theoretical properties, including stability, invariance to representation reparameterization under specified conditions, and connections to existing concept-editing formulations. Experiments show improved task preservation during erasure and improved concept recoverability in some transfer settings.

Critical strengths

The paper offers a topological perspective on relationships between concepts and applies it to practical concept interventions. The improvements in worst-group accuracy (WGA) on the erasure tasks are significant.

Critical weaknesses
  1. TCR does not show a clear advantage on the concept transfer task.
  2. TCR-DIS introduces a considerable preprocessing cost, according to Table 8.
Questions for authors
  1. How does the proposed topology-based approach differ theoretically from existing disentanglement methods, and what advantages does it offer? In Table 1, why does TCR-ERA outperform some TCR-DIS variants that explicitly model relationships between concepts?
  2. Could the authors evaluate TCR directly on established disentanglement benchmarks to clarify its advantage over existing methods?
5 comments

We thank the reviewer for their comments and questions, which we address point by point below.

"TCR does not show a clear advantage on the concept transfer task."

The improvements in concept transfer are indeed more moderate than those observed for concept erasure, particularly for ShuffleNetV2, where SPKD remains the strongest baseline. Nevertheless, the results provide encouraging evidence for the potential of TCR in this setting.

In particular, for ResNet8×4, TCR achieves the highest concept recoverability (60.05), improving by 5.54 points over the strongest baseline (SPKD, 54.51), and by 5.98 points over standard KD (54.07), while also achieving the highest top-1 accuracy (73.68). Beyond the concepts explicitly optimized, TCR also improves recoverability on independently discovered concept banks. For instance, at the finest spatial scale, optimizing 20 concepts improves recoverability on a separately discovered 10-concept bank by 4.94 points over SPKD (Fig. 4).

While these improvements do not establish a consistent advantage across architectures, we believe they demonstrate the promise of using a common concept-level abstraction for knowledge transfer, particularly for preserving information that conventional distillation objectives may overlook. We see this as an encouraging direction for further investigation.


"TCR-DIS introduces a considerable preprocessing cost, according to Table 8."

TCR-DIS indeed introduces an additional preprocessing cost, although its absolute runtime remains relatively moderate in several settings. For instance, combining TCR-DIS with lightweight methods such as LEACE or SPLINCE requires only 33–89 seconds in total across the four datasets (Table 8), including both preprocessing and erasure. For comparison, standalone RLACE requires 39–209 seconds and LEOPARD 11–132 seconds, with TCR-DIS+LEACE/SPLINCE being faster than both on three of the four datasets. More computationally intensive combinations can nevertheless be substantially slower, notably TCR-DIS+RLACE on DIAL-Mention (1001 seconds, compared to 209 seconds for RLACE alone). Thus, while the additional cost is not negligible, combining TCR-DIS with efficient erasure methods can remain computationally competitive with existing approaches, while providing substantial improvements in task preservation.


"How does the proposed topology-based approach differ theoretically from existing disentanglement methods, and what advantages does it offer?" and "Could the authors evaluate TCR directly on established disentanglement benchmarks to clarify its advantage over existing methods?"

A clearer comparison with existing disentanglement methods would indeed help position our contribution. Classical unsupervised methods, such as β-VAE [1], FactorVAE [2], β-TCVAE [3], and DIP-VAE [4], aim to discover separately encoded generative factors without supervision regarding which factors should be disentangled. Their objectives therefore differ substantially from our setting, where the concepts of interest are specified and the goal is to modify their relationships within a pretrained representation.

Although different, supervised disentanglement methods share more similarities, as they use labels to guide the separation of specified factors, that could be interpreted as the concepts of interest in our setting. These methods typically aim to learn representations with separated factors directly from the data, rather than reorganize concepts already encoded in a pretrained model. While they could be adapted to operate on pretrained features, this is not their primary setting. Methods such as DIVA [5], CCVAE [6], Siddharth et al. [7], CSVAE [8], F-Statistic Loss [9], and DSN [10] encourage, through different mechanisms, factor information to be represented in separate components (i.e. subspaces) of the optimized representation. However, encouraging each factor to be encoded in a dedicated subspace does not necessarily ensure its information from not being encoded in other subspaces. Consequently, erasing a concept in a subsequent step may still substantially affect the recoverability of another, whereas TCR-DIS directly optimizes this interference, being the targeted use-case in our work. Other methods, such as DiCyR [11] and LORD [12], isolate a single supervised factor from complementary information, but do not explicitly protect the recoverability of a second specified factor, which may consequently be damaged during disentanglement or subsequent erasure, contrary to TCR-DIS. Guided-VAE [13] goes further by adversarially discouraging factor information outside their designated components, but targets a broader statistical separation than required for selective erasure, potentially introducing unnecessary optimization constraints when applied in our use-case, whereas TCR-DIS directly targets erasure-induced interference.

continuation in the next comment

continuation of the previous comment

Moreover, many of these methods require enforcing factor-specific latent components and explicitly specifying their dimensions. In contrast, TCR-DIS does not prescribe how concept information should be distributed across the representation. Instead, it characterizes disentanglement through the interference rC→Y=1−mY(EC(Z))/mY(Z)r_{C\rightarrow Y}=1-m_Y(\mathcal E_C(Z))/m_Y(Z), measuring how much erasing CC reduces the recoverability of YY. By reducing this interference while preserving individual concept recoverability, TCR-DIS directly targets our intended intervention behavior, without requiring statistical independence or a predefined factorization. We will add these clarifications in the related work of the revised manuscript.

Beyond this distinction in disentanglement objectives, TCR provides a general representation of concepts and their interactions. Rather than designing a dedicated formulation for each intervention, different objectives, including concept erasure, disentanglement, and transfer, can be expressed by specifying different target concept organizations within the same optimization framework. Persistence provides the underlying topological abstraction, characterizing how concepts connect across different interaction levels and identifying the relationships responsible for merging concept groups. These relationships can then be selectively targeted and updated as the concept organization evolves during optimization.

Regarding established disentanglement methods and benchmarks, we would be interested in better understanding which ones the reviewer was indeed thinking about. Could the reviewer clarify whether they have in mind classical unsupervised methods, such as β-VAE and FactorVAE, or supervised approaches, such as DIVA, CCVAE, and Guided-VAE? Similarly, are they referring to benchmarks such as dSprites, Shapes3D, MPI3D, or CelebA?

References

[1] Higgins et al. (2017). β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. ICLR.

[2] Kim & Mnih (2018). Disentangling by Factorising. ICML.

[3] Chen et al. (2018). Isolating Sources of Disentanglement in Variational Autoencoders. NeurIPS.

[4] Kumar et al. (2018). Variational Inference of Disentangled Latent Concepts from Unlabeled Observations. ICLR.

[5] Ilse et al. (2020). DIVA: Domain Invariant Variational Autoencoders. MIDL.

[6] Joy et al. (2021). Capturing Label Characteristics in VAEs. ICLR.

[7] Siddharth et al. (2017). Learning Disentangled Representations with Semi-Supervised Deep Generative Models. NeurIPS.

[8] Klys et al. (2018). Learning Latent Subspaces in Variational Autoencoders. NeurIPS.

[9] Ridgeway & Mozer (2018). Learning Deep Disentangled Embeddings With the F-Statistic Loss. NeurIPS.

[10] Bousmalis et al. (2016). Domain Separation Networks. NeurIPS.

[11] Bertoin & Rachelson (2024). Disentanglement by Cyclic Reconstruction. IEEE TNNLS.

[12] Gabbay & Hoshen (2020). Demystifying Inter-Class Disentanglement. ICLR.

[13] Ding et al. (2020). Guided Variational Autoencoder for Disentanglement Learning. CVPR.


"In Table 1, why does TCR-ERA outperform some TCR-DIS variants that explicitly model relationships between concepts?"

TCR-ERA and TCR-DIS address related but distinct objectives, which may explain their differences in performance. Although TCR-ERA does not explicitly model concept interactions, it directly optimizes the removal of the unwanted concept while preserving task recoverability, allowing it to achieve a favorable trade-off between the two. In contrast, TCR-DIS first reduces interference between concepts while preserving their individual recoverability, before applying a separate erasure method. Its main benefit is therefore to provide a general preprocessing step that can facilitate subsequent erasure using different methods, rather than directly optimizing the final erasure outcome.

Importantly, TCR-ERA does not systematically outperform TCR-DIS. For instance, TCR-DIS combined with the best-performing erasure method achieves higher WGA on Waterbirds (74.66 vs. 71.17), DIAL-Mention (58.20 vs. 55.70) and Jigsaw (61.05 vs. 57.37), whereas TCR-ERA performs better on Bias in Bios (86.52 vs. 83.50). These results highlight the complementary roles of both approaches: TCR-ERA offers effective direct erasure, while TCR-DIS provides a reusable preprocessing step that can improve diverse existing erasure methods.


We hope that our responses and the additional evidence provided address your main concerns. We would be happy to clarify any remaining questions and welcome your reassessment in light of these clarifications.

Area_Chair_EFvsOpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Reviewer_AWZFOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Topological Concept Representations (TCR), an abstraction over the concepts represented by a neural network and the relationships between them. The method allows modifications to the abstraction to be propagated to the underlying neural network representation. Stability and reparametrization-invariance properties of TCR are proved, and experiments show that it can be used to improve concept erasure performance and to transfer concept information from one model to another.

Critical strengths
  1. The idea of abstracting concept organisation as a persistence diagram over concept recoverability and interaction scores, and using edits to that diagram as a unified interface for erasure, disentanglement and transfer, is, to my knowledge, a novel and interesting idea.

  2. Clear gains over concept erasure baselines are shown in the experiments.

Critical weaknesses
  1. In some situations, concepts may be highly correlated. In this case, what does it mean to "adjust" these concept relationships? If concepts are highly correlated, surely it will always be possible to predict one from information about the other?

  2. The related work section in the appendices is slightly thin and could be included in the main paper. The paper states that existing approaches do not provide a "common, editable representation of concepts", but I believe that Concept Bottleneck Models (CBMs) [1] and related approaches may? How does this work compare to CBMs [1] and related models [2]? Prior work has also looked at concept structure and relationships [3].

  3. I have a few questions about the results. Why is worst-group accuracy reported in Table 1 rather than overall accuracy? Would it be possible to also include overall accuracy results? What does "with an adjusted balanced linear concept leakage <10%" mean? On line 460, are any measures taken to ensure the two sets of discovered concepts do not overlap by chance?

  4. The paper does not contain any discussion of the computational complexity of the method.

# Minor Issues

These minor issues did not affect my rating and do not need to be addressed in the rebuttal (although they could be fixed in a future iteration of the manuscript).

  • "of" on line 59 should be "to".
  • I believe this may be a typo (line 284): "Given two sets of concepts [...], we defined their structural disentanglement by lowering the persistence saddles".
  • On line 425, "teacher" should be "teacher's".

# References

[1] Koh, P.W., Nguyen, T., Tang, Y.S., Mussmann, S., Pierson, E., Kim, B. and Liang, P., 2020, November. Concept bottleneck models. In International conference on machine learning (pp. 5338-5348). Pmlr.

[2] Espinosa Zarlenga, M., Barbiero, P., Ciravegna, G., Marra, G., Giannini, F., Diligenti, M., Shams, Z., Precioso, F., Melacci, S., Weller, A. and Lio, P., 2022. Concept embedding models: Beyond the accuracy-explainability trade-off. Advances in neural information processing systems, 35, pp.21400-21413.

[3] Hill, O., Espinosa Zarlenga, M. and Jamnik, M., 2026, April. Hierarchical concept-based interpretable models. In International Conference on Learning Representations (Vol. 2026, pp. 11581-11609).

Questions for authors
  1. Does it make sense to "adjust" the relationship between concepts that are highly correlated?
  2. How does the paper compare to the prior work mentioned in point 4 above?
  3. What is the computational complexity of TCR? How long does it take to run?
6 comments

We thank the reviewer for their comments and questions, which we address point by point below.

"In some situations, concepts may be highly correlated. In this case, what does it mean to "adjust" these concept relationships? If concepts are highly correlated, surely it will always be possible to predict one from information about the other?" and "Does it make sense to "adjust" the relationship between concepts that are highly correlated?"

By adjusting concept relationships, TCR does not seek to modify the statistical correlation between concept labels, but rather how their information is encoded and interacts within the learned representation. Specifically, Eq. 3 measures how much erasing one concept reduces the recoverability of another. TCR-DIS seeks to reduce this interference, making subsequent erasure less damaging to task-relevant information. This remains meaningful for correlated concepts, which may still contain partially distinct predictive information.

We agree, however, that correlations impose fundamental limits. For instance, if the task and concept labels are identical, perfect erasure and task preservation cannot both be achieved. More generally, preserving task information may retain indirect predictability of the unwanted concept. In particular, the linear guardedness considered by LEACE does not guarantee statistical independence or prevent nonlinear concept recovery, as further illustrated by our conditional and nonlinear leakage evaluations in Appendix I.1.1.

Thus, our results demonstrate improved task preservation at comparable linear concept leakage, rather than complete statistical disentanglement. We will clarify this distinction and its limitations in the main text of the revised manuscript.


"The related work section in the appendices is slightly thin and could be included in the main paper. The paper states that existing approaches do not provide a "common, editable representation of concepts", but I believe that Concept Bottleneck Models (CBMs) [1] and related approaches may? How does this work compare to CBMs [1] and related models [2]? Prior work has also looked at concept structure and relationships [3]." and "How does the paper compare to the prior work mentioned in point 4 above?"

Concept Bottleneck Models (CBMs; Koh et al., 2020) provide an editable concept-level interface for prediction, while Concept Embedding Models (CEMs; Espinosa Zarlenga et al., 2022) extend this approach with richer concept embeddings. Hierarchical CEMs of Hill et al. (2026) explicitly model semantic parent–subconcept relationships and support interventions at different levels of granularity.

While these approaches enable concept-level interventions, their primary objective is to explain and improve predictions through a concept-based architecture. In contrast, TCR operates post-hoc on arbitrary pretrained representations, without requiring predictions to pass through a concept bottleneck. It characterizes concept recoverability and intervention-induced interactions, whose hierarchical organization is captured through persistence, rather than predefined semantic relationships. Importantly, TCR interventions modify how concepts are encoded and interact within the representation itself, rather than correcting concept states or adjusting their contributions to predictions within a concept-based model.

We agree that these works provide important complementary perspectives. We will expand the related work discussion in the main text of the revised manuscript and clarify our positioning relative to these approaches.


"I have a few questions about the results. Why is worst-group accuracy reported in Table 1 rather than overall accuracy? Would it be possible to also include overall accuracy results?"

We primarily report worst-group accuracy (WGA), a standard metric in spurious-correlation benchmarks, because concept erasure may disproportionately affect certain task–concept groups. In particular, when groups are imbalanced (see Appendix H.1), overall accuracy can mask substantial performance degradation on underrepresented groups. WGA instead measures performance on the most affected group, providing a more informative assessment of task preservation across groups.

Nevertheless, we agree that overall accuracy provides a useful complementary perspective and that this could be mentioned in the main text. These results are reported in Appendix I.1 (Table 8), showing that TCR also improves overall task accuracy in several settings. Across all datasets and baselines, TCR-DIS and TCR-ERA improve overall accuracy by 4.31 and 6.99 percentage points on average, respectively, further demonstrating that their benefits extend beyond WGA. We will explicitly reference these results in the main text of the revised manuscript.


"What does "with an adjusted balanced linear concept leakage <10%" mean?"

We agree that this could be made clearer in the main text. As defined in Appendix H.1, concept leakage is evaluated using a linear logistic-regression probe trained to predict the unwanted concept from the edited representation. We measure its adjusted balanced accuracy, defined as Leak=2,BalAcc−1\mathrm{Leak}=2\\,\mathrm{BalAcc}-1, where BalAcc\mathrm{BalAcc} averages the recalls of the two concept classes. Thus, a leakage of 0 corresponds to chance-level prediction and 1 to perfect prediction, accounting for class imbalance. The threshold of 10%10\% therefore corresponds to a balanced accuracy below 5555\\%, i.e., within 5 percentage points of chance level (5050\\%). For model selection, we enforce this constraint on both training and validation data, allowing us to compare task preservation across methods at similarly low linear concept leakage. We will make this definition more explicit in the main text of the revised manuscript.


"On line 460, are any measures taken to ensure the two sets of discovered concepts do not overlap by chance?"

Concept banks are independently fitted using Dictionary Learning with non-negative sparse codes, without explicitly enforcing non-overlap. To quantify their overlap, we measure, for each concept in the smaller bank, its maximum absolute correlation with concepts in the larger bank, and average these values. We observe moderate to strong correlations (0.38–0.84), which are expected since the banks are learned from the same teacher representation.

Smaller bankLarger bankNo cropCrop 2×2Crop 3×3Crop 4×4
250.590.680.630.70
2100.560.640.570.54
2200.380.540.470.53
5100.840.740.770.74
5200.670.640.650.67
10200.840.840.760.80

Importantly, this overlap does not appear sufficient on its own to explain the main trend observed in Fig. 3: finer-scale concepts exhibit stronger cross-bank transfer despite similar or even lower correlations. For instance, the correlation between the 5- and 10-concept banks decreases from 0.838 (full images) to 0.743 (4×4 crops), while cross-bank transfer generally improves with finer spatial scales. Thus, although shared information may contribute to transfer, these observations remain consistent with our interpretation in Sec. 4.2 that concept locality also plays an important role.

We will clarify in the revised manuscript that independently discovered banks are not necessarily composed of independent concepts.


"The paper does not contain any discussion of the computational complexity of the method." and "What is the computational complexity of TCR? How long does it take to run?"

We agree that it would be better to mention this in the main text. The computational complexity is analyzed in Appendix C. For NN samples of dimension dd and nCn_C concepts, constructing TCR using the analytical MSE formulation requires O(Nd2+NdnC+d3+dnC2)O(Nd^2+Ndn_C+d^3+dn_C^2) operations. Importantly, all concept scores and pairwise interactions share a single covariance decomposition, avoiding separate decompositions for each concept pair. With TT Newton iterations per probe, the interference-based formulation has a total complexity of O(Nd2+NdnC+d3+nC2T(Nd2+d3))O(Nd^2+Ndn_C+d^3+n_C^2T(Nd^2+d^3)), assuming shared covariance preprocessing and a dense Newton solver, used only for the two-concept erasure experiments. For concept transfer, we further reduce the computational cost by incrementally updating statistics from a bounded feature bank.

Regarding runtime, Appendix I.1 provides execution times for all erasure methods. Direct TCR-ERA takes 4.05–142.63 seconds across datasets, making it faster than RLACE on all four datasets and LEOPARD on three. TCR-DIS introduces additional preprocessing overhead, but its combination with lightweight methods such as LEACE and SPLINCE remains below 90 seconds. More expensive combinations, such as TCR-DIS with RLACE, can require substantially longer (up to 1001 seconds). Overall, these results show that TCR can achieve substantial improvements in task preservation at a moderate computational cost, particularly when combined with efficient erasure methods. We will summarize these results and reference the corresponding appendix sections in the main text of the revised manuscript.


""of" on line 59 should be "to". I believe this may be a typo (line 284): "Given two sets of concepts [...], we defined their structural disentanglement by lowering the persistence saddles". On line 425, "teacher" should be "teacher's"."

We will fix these mistakes in the revised manuscript.


We hope that our responses and the additional evidence provided address your main concerns. We would be happy to clarify any remaining questions and welcome your reassessment in light of these clarifications.

Area_Chair_EFvsOpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Reviewer_pjVuOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes Topological Concept Representations (TCR), a post-hoc operational abstraction that describes the importance of the encoded concepts and how they interact in a given neural network representation. For each concept and for each pair of concepts, TCR constructs a concept mesh and defines intervention over concepts as a transformation over this graph representation. The author claims this is a unified framework for concept intervention, disentanglement and concept transfer.

Critical strengths

The paper proposes two interesting ideas:

  • The idea of providing a unified framework for intervention over concepts is a valuable idea, especially because "interventions" over concept-based models are not grounded in causality terms, but rather inspired by it.

  • Moreover, the idea of modeling concepts over topology is valuable, as many concept-based methods rely on the independence assumption, i.e. they threat concepts independently.

Critical weaknesses

The main weakness is certainly clarity: while the main idea and intuition is clear, the details about the implementations, the experiments and the design choices of the method are not adequately justified. Unfortunately, many of main point of the papers are unclear to me.

Below I will mention in detail this concerns about clarity, and expand them:

  1. The problems this paper tries to address are never formalized, this makes it hard to evaluate whether the method is valuable or not. Let me explain this in detail: the main claim of the paper is this "common framework" for concept interventions, but this is never formalized explicitly (I am referring to the problem, not what TCR does), i.e. there no definition of an intervention, of concept organization or of concept leakage. Another example is the concept of abstraction, on which the paper builds on. It is not clear what an abstraction is neither from the introduction nor from the preliminaries. The terminology suggests a connection to causal abstraction (e.g., Geiger et al., "Causal Abstractions of Neural Networks"), where an abstraction is a map between the representation of a neural network and a structural causal model together with a consistency condition under interventions, but this is never made explicit, even in the related work.

  2. Mathematical formalism is too heavy, without the need to. Section 2 introduces simplicial complexes, lower-star filtrations, symbolic perturbation, the filtration map, the persistence map. Not all of this is needed for the reader to understand the method, but rather, it causes confusion. I feel like this misses the main intuition underline the mathematical object as well as some running examples. To ground my doubts, just as representative examples, the Wasserstein distance is introduced but never evaluated in the experiments, same with the different concept-scores that could have been described in few words (with details in the appendix), or almost all section 3.3 which is not clear the role it plays in the paper.

  3. The provided theory seem not to play a relevant role in the experiment. To me, the whole ection 3.4 does not give additional insights to the reader and it is not clear what role it plays in the evaluated experiment.

  4. Experiments lack many details. As far as I understood, there are several components that play a role in TCR, that are a probe (unclear whether it is linear or not), the matrix construction and the transformation. In none of the experiments it is clear how this are constructed, whether you supervise something or elsewhere, what concept did you use, how you test your models. Furthermore, no experiment isolates the contribution of the topological abstraction, which is the main claim of the paper. Additionally, it is not clear how you tested leakage. Without understanding this details it is hard for me to evaluate what TCR can do precisely and whether it can meet its claims.


My second main concern is that the authors have put the related work in the appendix, I suggest that, in case of acceptance, they will be put in the main paper.

Questions for authors

Here are my question to the authors:

  1. Can you give a formal problem statement of each problem TCR addresses and how it addresses them?
  2. In what sense is TCR an abstraction? What is the connection with causal abstractions (Geiger et al.)?
  3. Why did you not use a covariance as for s definition? It is unclear to me why you odel s_{ij} as minimum.
  4. From the paper it is unclear why you need such a convoluted construction to model concept interaction and intervention. Can you explain me the intuition behind each component, and why precisely you use a persistence diagram as topological representation?
  5. Can you make clear the setting of the experiments? Metrics, training and setting.
6 comments

We thank the reviewer for their comments and questions, which we address point by point below.


W1.1 and Q1: Formal Problem Statement

We agree that the general problems motivating TCR could have been formalized more explicitly, independently of our proposed framework.

We consider the following representation-level concept intervention problem: given a learned representation HH and concepts C\mathcal C, find a transformation TθT_\theta that achieves desired changes in their encoding and relationships, subject to application-specific constraints. Formally, concept organization comprises individual concept recoverability scores mi(H)m_i(H) and pairwise interactions ρij(H)\rho_{ij}(H). An intervention is a transformation Z=Tθ(H)Z=T_\theta(H) intended to modify selected values of mi(Z)m_i(Z) or ρij(Z)\rho_{ij}(Z) while potentially preserving others.

This encompasses three applications, whose realization through TCR is partially described in Sec. 3.3:

  1. Concept erasure: Minimize the recoverability mC(Z)m_C(Z) of an unwanted concept CC, subject to preserving task information, e.g., mY(Z)≥mY(H)−εm_Y(Z)\geq m_Y(H)-\varepsilon. TCR-ERA addresses this by lowering the corresponding concept maximum while preserving selected maxima.
  2. Concept disentanglement: Minimize the interaction ρij(Z)\rho_{ij}(Z) between selected concepts while preserving their recoverability, e.g., mi(Z)≈mi(H)m_i(Z)\approx m_i(H) and mj(Z)≈mj(H)m_j(Z)\approx m_j(H). TCR-DIS addresses this by lowering interaction saddles while preserving concept maxima, notably to facilitate subsequent erasure.
  3. Concept transfer: Given teacher and student representations HTH_T and HSH_S, minimize discrepancies between their concept organizations while maintaining task performance. TCR realizes this by matching their persistence diagrams alongside standard distillation objectives.

The common framework supports both different intervention objectives and different criteria for concept recoverability and interactions. TCR expresses the desired changes through target topological representations and a shared optimization objective (Sec. 3.3). Its flexibility is further supported by Sec. 3.4 and Appendix D, which recover several existing erasure criteria as particular formulations.

Finally, concept leakage refers to the recoverability of an unwanted concept from the transformed representation, under a specified prediction criterion. Formally, it can be quantified as the improvement in prediction loss over an optimal constant predictor, as in our definition of mC(Z)m_C(Z) (Eq. 1). Experimentally, we primarily evaluate leakage through adjusted balanced linear-probe accuracy (Appendix H.1), complemented by nonlinear and conditional evaluations (Appendix I.1.1).


W1.2 and Q2: Abstraction Definition

We agree that the notion of abstraction could be more explicitly defined. We use this term in the sense of a compact, higher-level representation that retains and extracts selected properties of an underlying object while discarding other details. Specifically, TCR maps a learned representation to a topological summary of concept recoverability and interactions. Through persistence, it retains the interactions determining the multiscale connectivity of concepts, rather than all pairwise relationships. We call this abstraction operational because desired modifications can be specified at this higher level and translated into representation edits through optimization.

This differs from causal abstractions (Geiger et al., 2021), which seek to establish that interventions on neural representations produce the same effects as corresponding interventions on a high-level causal model. TCR does not construct a causal model or claim such interventional consistency. Although its interaction scores can be defined through the effects of concept erasure, they characterize representational relationships rather than causal dependencies.

We will clarify the meaning of abstraction in the introduction and explicitly distinguish TCR from causal abstraction in the revised manuscript.

W2: Heavy Mathematical Formalism

Sec. 2 indeed introduces substantial mathematical formalism, which may make the main ideas more difficult to follow. These elements are motivated by the need to formally define the persistence-based optimization framework underlying TCR. In particular, the filtration and persistence maps are essential for explaining how gradients propagate from the topological abstraction back to the learned representation. Nevertheless, we agree that not all these technical details need to appear in the main text. We will revise the presentation to emphasize the intuition behind these constructions, moving some formal details to the appendix and considering a simple running example to illustrate the method.

Regarding the specific points raised, the Wasserstein distance is not an evaluation metric, but rather the central optimization objective of TCR (Eq. 5). It measures the discrepancy between the current and target persistence diagrams, allowing us to translate desired concept-level changes into an optimization problem. Similarly, Sec. 3.3 constitutes the core of our intervention framework: it explains how different interventions are expressed as modifications of a target TCR, and how these modifications are propagated back to the underlying representation through differentiable topological optimization.

We will revise Secs. 2 and 3 to better distinguish the essential intuitions from the technical machinery needed to formally establish the framework.


Q4: Intuition behind TCR Components

The motivation behind the different components of TCR could indeed be explained more intuitively.

Our goal is to construct an abstraction that captures both which concepts are encoded in a representation and how they interact. To this end, concept scores measure how strongly each concept is encoded, while interaction scores characterize how their encodings depend on one another. We combine these quantities into a concept mesh, where concepts form maxima and their interactions form saddles, allowing us to describe their organization across different interaction levels.

The motivation for using persistence diagrams is to capture how these local interactions collectively determine the global organization of concepts. Rather than retaining every pairwise interaction, persistence identifies the interactions responsible for merging groups of concepts, providing a compact, multiscale representation of their connectivity. This allows us to focus interventions on the interactions that determine this organization, while benefiting from established tools for comparing and optimizing persistence diagrams through Wasserstein distances. This is the most used topological representation and benefits from a variety of established tools compared to more advanced topological representations (such as merge trees).

Importantly, this abstraction provides a common interface for different interventions: erasure, disentanglement and transfer can all be expressed as modifications of a target TCR, without requiring separate optimization frameworks.

We will improve the presentation of Sec. 3 to better convey these intuitions and the motivation behind each component.


W3: Relevance and Impact of the Proven Theoretical Properties

We agree that the practical significance of the theoretical properties in Sec. 3.4 could be better explained. While these results do not directly establish the empirical improvements observed in our experiments, they provide important insights into why TCR constitutes a meaningful abstraction for concept-level interventions.

First, the persistence reduction retains the interactions responsible for merging concept components across scales, while discarding redundant pairwise relationships. This provides a principled and compact characterization of concept organization, focusing optimization on the interactions determining its connectivity rather than treating every pair independently. Second, we show that several existing concept-erasure criteria can be recovered as specializations of TCR at the objective or constraint level (Appendix D), supporting our motivation for a common framework that accommodates different intervention objectives. Finally, for the analytical MSE–overlap formulation used in concept transfer, we establish invariance to invertible affine reparameterizations and local stability under small representation perturbations. These properties ensure that TCR does not depend on arbitrary choices of feature coordinates and that its structure cannot change arbitrarily under sufficiently small perturbations, under the stated regularity conditions.

Together, these properties justify the relevance of TCR as an abstraction for the erasure, disentanglement and transfer applications evaluated experimentally. We will revise Sec. 3.4 to better explain the intuition behind each property and clarify its connection to the experimental applications.

"Why did you not use a covariance as for s definition?"

Interestingly, in addition of the interference saddle, we also consider another interaction based on covariance through predictive overlap (Appendix B). Specifically, it measures the squared correlation between the optimal linear predictions of two concepts, derived from the representation covariance ΣZ=Cov⁡(Z)\Sigma_Z=\operatorname{Cov}(Z) and the concept–representation cross-covariances gi=Cov⁡(Z,ci)\mathbf{g}_i=\operatorname{Cov}(Z,c_i). Unlike the covariance between concept labels Cov⁡(ci,cj)\operatorname{Cov}(c_i,c_j), which remains unchanged when editing the representation and therefore provides no optimization signal, predictive overlap characterizes how concepts are jointly encoded in the representation and responds to its modifications. Alternatively, our interference-based formulation (Eq. 3) measures interactions through the effect of erasing one concept on the recoverability of another. We use interference for disentanglement to directly capture the effects of concept erasure, and predictive overlap for transfer because its efficient analytical formulation scales to larger concept sets (see Appendix B).

"It is unclear to me why you odel s_{ij} as minimum."

One of the term is indeed a minimum, though, it is not the only term, specifically we defined sij=min⁡(mi,mj)ρijs_{ij}=\min(m_i,m_j)\rho_{ij}, where ρij∈[0,1]\rho_{ij}\in[0,1] quantifies the interaction between concepts. In our experiments, this interaction is characterized either through the effect of erasing one concept on the recoverability of another (Eq. 3), or through the overlap of their predictive directions (Appendix B). The minimum serves a different purpose: it places interactions on the same scale as concept recoverability while ensuring sij≤min⁡(mi,mj)s_{ij}\leq\min(m_i,m_j).

This is essential to our topological construction, as bounding each interaction below both concept scores ensures that concepts form maxima and their interactions form saddles, allowing their relative prominence to be characterized through persistence. The minimum provides the largest admissible scaling factor: when the interaction is maximal, the saddle reaches the score of the weaker concept, giving it zero persistence; conversely, weaker interactions lower the saddle, increasing the persistence gap and reflecting a greater degree of separation between the concepts.

We will clarify the distinct roles of the interaction score and its scaling factor in the revised manuscript.


W4 and Q5: Experimental Setup

We agree that the experimental setup could have been presented more clearly in the main text. While detailed protocols are provided in Appendix H, we summarize the main elements here.

Concept erasure and disentanglement (Appendix H.1). We use four datasets with task/unwanted concept pairs: Waterbirds (bird/background), Bias in Bios (profession/gender), DIAL-Mention (mention/dialect) and Jigsaw (toxicity/identity). Representations are extracted from frozen pretrained ResNet-50 or BERT-base models and reduced to 128 dimensions using PCA. TCR uses linear logistic-regression probes to measure concept recoverability and LEACE-based interference to construct the concept mesh. We optimize a two-layer MLP to disentangle or erase the unwanted concept while preserving the task. Evaluation uses independent linear probes, reporting test worst-group accuracy (WGA) under comparable concept leakage, measured as adjusted balanced accuracy (2,BalAcc−12\\,\mathrm{BalAcc}-1). Additional non-linear and conditional leakage evaluations are provided in Appendix I.1.1.

Concept transfer (Appendix H.2). On CIFAR-100, concepts are discovered from a pretrained ResNet32×4 teacher through Dictionary Learning. TCR uses linear least-squares probes and predictive-overlap interactions, and transfers the resulting concept organization to ResNet8×4 and ShuffleNetV2 students through a distillation objective. We evaluate test top-1 accuracy and concept recoverability, measured through held-out linear probes predicting teacher concept coefficients.

Topological contribution. We agree that isolating the contribution of the topological abstraction would strengthen the evaluation. We are currently setting up an ablation isolating the contribution of TCR. We will share the results with the reviewer as they become available.

Given the space constraints, we believe the appendix is the appropriate place for the complete experimental protocols. Nevertheless, we will improve the main-text summaries and references to these sections to make the experimental setting clearer and easier to follow.


Missing Related Work in Main Text

We will move the related work in the main text of the revised manuscript.


We hope that our responses and the additional evidence provided address your main concerns. We would be happy to clarify any remaining questions and welcome your reassessment in light of these clarifications.

Reviewer_szh9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Topological Concept Representations (TCR), a post-hoc abstraction that summarizes which concepts are encoded in a representation and how they interact. Concept recoverability scores and pairwise interference scores are placed on a concept graph and summarized as a persistence diagram; interventions such as erasure, disentanglement, and transfer are specified by editing this diagram and optimizing a transformation of the representation to match it. Experiments show improved worst-group accuracy when TCR disentanglement is used before existing erasure methods on four datasets, and improved concept recoverability in teacher–student distillation on CIFAR-100.

Critical strengths

S1. Framing different concept-level interventions as edits to a shared abstraction is a novel and well-motivated idea.

S2. Under the same linear leakage constraint, TCR disentanglement consistently improves worst-group accuracy across seven erasure methods and four datasets.

S3. The appendix provides useful additional analyses, including non-linear and oracle-conditioned leakage, representation similarity, and runtime, and the limitations are discussed openly.

Critical weaknesses

Critical Weaknesses

W1. The contribution of the topological component could be made clearer. In the erasure experiments, the concept graph appears to contain only the task and concept nodes (Appendix C, H.1), so the persistence diagram is relatively simple. An ablation that directly optimizes the concept and interaction scores without the persistence reduction would help clarify the added value of topology.

W2. TCR-DIS uses a non-linear MLP trained with task labels, while most baselines are linear. A comparison with the same MLP trained with a non-topological objective would help separate the effect of TCR from the flexibility of the transformation.

W3. The leakage constraint is linear only. Based on Table 11, TCR-DIS does not consistently reduce non-linear leakage and increases it in some settings. Further discussion of how this affects the claim of improved utility at comparable leakage would be helpful.

W4. The transfer gains are relatively modest: on ResNet8×4, the top-1 improvement is close to the reported standard deviation, and on ShuffleNetV2, SPKD performs better. Including a more direct baseline (e.g., regressing teacher concept coefficients) or CRD would make the comparison more complete.

Minor Weaknesses

m1. Some baselines appear to report identical results (e.g., LEACE, SPLINCE, and LEOPARD on Waterbirds and Jigsaw); a brief explanation would be helpful.

m2. The theoretical results appear to focus on the MSE–overlap instantiation; their relation to the setting used in the erasure experiments could be clarified.

m3. The average WGA gain is largely driven by Waterbirds, with smaller gains on the text datasets.

m4. There are a few typos and grammatical errors, e.g., "permits to express" (Section 3.1), "Eq. equation 14" (Appendix E.2), and "In addition of" (Appendix I.1).

Questions for authors

Q1. How does TCR compare with directly optimizing the concept and interaction scores without the persistence reduction? In settings with few nodes, what does the topological step add? (see W1)

Q2. Could you report results for the same MLP trained with a non-topological objective? (see W2)

Q3. How should the increase in non-linear leakage in some settings be interpreted? (see W3)

Q4. What explains the identical results of some baselines? (see m1)

5 comments

We thank the reviewer for their comments and questions, which we address point by point below.


W1 and Q1: Benefits of the persistence reduction

The two-concept erasure experiments indeed represent a minimal setting, where there is only one interaction between concepts and the reduction made by persistence is therefore minimal. Nevertheless, it allows these interventions to benefit from established tools for comparing and optimizing topological representations, notably Wasserstein distances and differentiable persistence-based optimization. More importantly, this simple setting constitutes one particular application of a common framework supporting different interventions, including erasure, disentanglement and concept transfer, as well as more complex concept organizations.

The benefits of the persistence reduction become increasingly pronounced as the number of concepts grows. Rather than imposing constraints on every pairwise interaction, persistence retains only those necessary to characterize how concepts progressively organize into groups. Interestingly, these interactions form a maximum-spanning-tree backbone, which retains a strongest interaction across every possible partition of the concepts into two groups. This provides a principled way to preserve the strongest relationships between concept groups without requiring every individual interaction to be reproduced. In particular, when optimization capacity is limited, focusing on these structurally important interactions may avoid additional constraints that are redundant or difficult to satisfy simultaneously. Moreover, as the representation evolves, the interactions selected by persistence adapt accordingly.

We agree that an ablation comparing persistence-based optimization with direct optimization of concept and interaction scores would help establish these benefits experimentally. We are currently setting up such an experiment to isolate the contribution of topology and will share the results with the reviewer as they become available. We will also clarify these motivations and the distinction between the minimal and general settings in the revised manuscript.


W2 and Q2: Contribution of the Nonlinear Transformation

TCR-DIS indeed uses a nonlinear MLP, whereas several erasure baselines rely on linear transformations. However, other baselines, notably KRaM and Obliviator, also use nonlinear MLPs, for which we use the exact same architecture as TCR-DIS in our experiments. The improvements observed over these methods therefore suggest that the benefits cannot be attributed to the MLP architecture alone, but also depend on the objective being optimized, particularly TCR-DIS's explicit reduction of concept interference.

To better isolate this contribution, we are also preparing an additional ablation to better isolate the contribution of explicitly optimizing concept interaction saddle, by comparing TCR-DIS with the same MLP architecture using a non-topological objective that does not optimize the interaction saddle. While the ablation discussed in W1/Q1 will investigate the specific benefits of persistence reduction, this experiment will help determine whether optimizing concept topological saddles provides benefits beyond the nonlinear transformation and task supervision alone. We will share the results with the reviewer once available.


W3 and Q3: Linear and Nonlinear Concept Leakage

Our primary evaluation indeed follows the linear-guardedness setting of established concept-erasure methods, such as LEACE, where concept leakage is assessed through linear probes. Consequently, our reported improvements in task preservation are established at comparable linear concept leakage, rather than under a guarantee of complete concept removal.

As shown in Table 11, TCR-DIS does not systematically reduce nonlinear leakage, which can indeed increase after disentanglement. This indicates that reducing interference measured through linear recoverability does not necessarily eliminate information accessible to nonlinear predictors. In particular, preserving task-relevant information may also retain information about a correlated unwanted concept. We therefore do not interpret our results as guaranteeing statistical independence or protection against nonlinear concept recovery.

Nevertheless, the additional evaluations also reveal encouraging results. In particular, TCR-DIS often substantially reduces oracle-conditioned leakage (Table 10), which measures concept recoverability when the task label is known. For instance, combining TCR-DIS with LEACE reduces this leakage from 33.37 to 10.24 on Jigsaw and from 72.35 to 60.76 on Waterbirds, while improving task preservation.

We will clarify in the revised manuscript that our main contribution is improved task preservation at comparable linear leakage, and discuss the remaining nonlinear leakage as a limitation and an interesting direction for extending TCR to stronger erasure criteria.


W4: Concept Transfer Results and Additional Baselines

The improvements in concept transfer are indeed more moderate than those observed for concept erasure, particularly for ShuffleNetV2, where SPKD remains the strongest baseline. We also acknowledge that the improvements in top-1 accuracy are relatively modest. Nevertheless, the results provide encouraging evidence for the potential of TCR in this setting, particularly regarding the preservation of concept-level information.

In particular, on ResNet8×4, TCR achieves the highest concept recoverability (60.05), improving by 5.54 points over the strongest baseline (SPKD, 54.51), and by 5.98 points over standard KD (54.07). TCR also achieves the highest top-1 accuracy (73.68), although the improvement over KD (+0.33) is small relative to the reported variability. Beyond the concepts explicitly optimized, TCR also improves recoverability on independently discovered concept banks. For instance, at the finest spatial scale, optimizing 20 concepts improves recoverability on a separately discovered 10-concept bank by 4.94 points over SPKD (Fig. 4).

We agree that including direct concept coefficient regression and CRD would provide valuable additional comparisons, particularly to better assess the benefits of transferring concept-level information. While our experiments already include several established distillation methods, we acknowledge that these additional baselines would strengthen the evaluation.

Overall, although the results do not establish a consistent advantage across architectures, we believe they demonstrate the promise of using a common concept-level abstraction for knowledge transfer. We see this as an encouraging direction for further investigation.


m1 and Q4: Similar Results Across Erasure Baselines

The similar results of LEACE, SPLINCE and LEOPARD can be explained by their closely related erasure mechanisms.

LEACE and SPLINCE both guarantee linear concept erasure through projections with the same kernel, but differ in their choice of projection range, with SPLINCE additionally aiming to preserve task-relevant covariance. Interestingly, Holstege et al. (2025, Sec. 3.2) show that these methods yield identical predictions when retraining an unregularized linear classifier, since the preserved linear information is determined by the kernel rather than the range. Although our classifiers use regularization, this explains why their results can remain very similar.

For LEOPARD, we use the cascaded formulation proposed and suggested by Saillenfest and Lemberger (2025), which first applies LEACE-based linear erasure before optimizing an additional nonlinear density-matching criterion. In our experiments, this additional stage does not yield a measurable improvement in the selected WGA on Waterbirds and Jigsaw, leading to the same reported scores as LEACE. This does not necessarily imply that the resulting representations are identical.

Some small differences can still be noted between SPLINCE and LEACE/LEOPARD on Bias and Bios SN and between LEACE/SPLINCE and LEOPARD on Dial Mention.

We will clarify these relationships between the baselines in the revised manuscript.


m2: MSE-focused Theory Clarification

We will clarify in the main text of the revised manuscript that the theoretical results focus on the MSE-overlap instantiation.


m3: Dataset-wise WGA Improvements

The average WGA improvement is indeed particularly influenced by Waterbirds. However, TCR-DIS improves WGA for every erasure baseline across all four datasets. More specifically, averaging the improvements over the seven erasure methods separately for each dataset (Table 1), we obtain gains of +53.91 points on Waterbirds, +21.31 on Bias in Bios, +6.04 on DIAL-Mention and +6.30 on Jigsaw.

Thus, while the improvements are more moderate on DIAL-Mention and Jigsaw, they remain consistent across all considered methods. Notably, even excluding Waterbirds, TCR-DIS achieves an average improvement of +11.22 WGA points across the three text datasets. We will clarify these dataset-wise improvements in the revised manuscript to provide a more complete picture of the results.


m4: Typos and Grammatical Errors

We will fix these mistakes in the revised manuscript.


We hope that our responses and the additional evidence provided address your main concerns. We would be happy to clarify any remaining questions and welcome your reassessment in light of these clarifications.

Comments on the paper

2 comments

We thank all reviewers for their constructive feedback and for the helpful discussions throughout the rebuttal period. We have revised the manuscript to address their concerns and improve the clarity of our contributions. In particular, we have strengthened the presentation of TCR as a common framework for concept-level interventions, clarified the motivation and benefits of the topological abstraction, and improved the experimental descriptions and discussion of limitations.

Importantly, we have also conducted two additional ablation experiments, investigating (1) the benefits of persistence reduction compared with direct optimization of concept scores and interactions, and (2) the contribution of the topological interaction-saddle objective compared with architecture-matched, non-topological MLP baselines.

Together, these changes address the prior reviewer concerns and, we believe, considerably strengthen the contribution.

Below, we summarize the main modifications and indicate where they can be found in the revised manuscript.

  • Reviewer szh9

    • We have clarified the role of persistence reduction, distinguishing its reduction in the two-concept setting from its benefits for larger concept organizations, particularly in selecting structurally important interactions (Section 3.4 - "Benefits of the persistence reduction").
    • We have added an ablation comparing persistence-based optimization with direct optimization of concept and interaction scores to better isolate the contribution of topology (Appendix I.2.1).
    • We have added an ablation using the same MLP architecture with a non-topological objective, isolating the contribution of explicitly optimizing concept interactions from that of nonlinear transformation capacity (Appendix I.2.2).
    • We have clarified that our erasure results establish improved task preservation at comparable linear concept leakage, rather than guaranteeing complete concept removal, and discussed the limitations regarding nonlinear leakage (Section 4.1 - Q1 - second paragraph).
    • We have explained the similarities between LEACE, SPLINCE and LEOPARD, clarifying why some of their results are identical or very similar (Section 4.1 - Q1 - first paragraph).
    • We have clarified that the theoretical invariance and stability results concern the analytical MSE-overlap formulation (Section 3.4).
    • We have expanded the discussion of the erasure results with the average improvement when excluding Waterbirds influencing the most the average (Section 4.1 - Q1 - first paragraph).
    • We have corrected the reported typos and grammatical errors.
  • Reviewer pjVu

    • We have made the common framework for concept-level interventions more explicit as a central contribution, introducing a general problem formulation and definitions of concept organization, interventions and concept leakage and clarified the meaning of TCR as an operational abstraction (Section 3.1 - first paragraph).
    • We have explained the distinction of TCR from causal abstractions, and how the two frameworks could potentially complement each other (Appendix A and referred to in the related work on the main text).
    • We have simplified the mathematical presentation in Section 2, moving less essential technical details to the appendix B and emphasizing the intuition behind the construction.
    • We have better motivated the different components of TCR, including concept scores, interaction scores, persistence reduction, Wasserstein-based optimization and the formulation of different interventions through target TCRs (Section 3.2 - first paragraph).
    • We have expanded the discussion of the theoretical properties, explaining their significance and relationship to the experimental applications (Section 3.4).
    • We have clarified the distinction between interaction measures and the scaling factor used to define persistence saddles, including the motivation for using the minimum of concept recoverability scores and the relationship with covariance-based measures (Section 3.2 - "From concept scores and interactions to topology." first paragraph).
    • We have improved the experimental descriptions in the main text, clarifying the concepts, probes, transformations, optimization objectives and evaluation metrics, with explicit references to the detailed protocols in the appendix (Section 4.1 - "Experimental Setup" and Section 4.2 - "Experimental Setup").
    • We have included the new ablation isolating the contribution of persistence reduction, also requested by Reviewer szh9, see the comment above (Appendix I.2.1).
    • We have moved the related work discussion into the main manuscript (End of Section 1).
  • Reviewer AWZF

    • We have clarified that TCR modifies representational interactions rather than statistical correlations between concept labels, and discussed the fundamental limitations of disentanglement and concept erasure for highly correlated concepts (Section 4.1 - Q1 - second paragraph).
    • We have expanded the related work discussion to better distinguish TCR from Concept Bottleneck Models, Concept Embedding Models and hierarchical concept-based approaches (Appendix A and referred to in the related work on the main text).
    • We have explicitly referenced in the main text the overall accuracy results already provided in the appendix, complementing the WGA evaluation (Section 4.1 - Q1 - first paragraph).
    • We have clarified the definition and interpretation of adjusted balanced linear concept leakage, including the meaning of the 10% threshold and how it is enforced during model selection (Section 4.1 - "Experimental Setup").
    • We have clarified that independently discovered concept banks may overlap and discussed the measured correlations between banks and their implications for interpreting cross-bank transfer (Section 4.2 - Q4).
    • We have summarized the computational complexity and runtime of TCR in the main text, with references to the detailed appendix analyses (end of Section 3.2 and second paragraph of Section 4.1 - Q1).
    • We have corrected the reported typos and grammatical errors.
  • Reviewer HqRY

    • We have clarified the computational overhead of TCR-DIS and its practical runtime relative to existing erasure methods, particularly when combined with lightweight methods such as LEACE or SPLINCE (Appendix J.1).
    • We have expanded the discussion of existing disentanglement approaches, including supervised and unsupervised methods, clarifying how TCR-DIS differs by directly targeting erasure-induced interference without requiring a factorization of the representation and further clarified the generality of TCR as a common framework for concept-level interventions, beyond its specific disentanglement application (Appendix A and referred to in the related work on the main text).
    • We have clarified the complementary objectives of TCR-ERA and TCR-DIS, explaining why direct erasure can sometimes outperform disentanglement followed by a separate erasure method (Section 4.1 - Q1 - first paragraph).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.