- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces Topological Concept Representations (TCR), an abstraction that jointly represents concepts and their relationships and provides a common interface for concept-level interventions. It provides theoretical properties, including stability, invariance to representation reparameterization under specified conditions, and connections to existing concept-editing formulations. Experiments show improved task preservation during erasure and improved concept recoverability in some transfer settings.
Critical strengths
The paper offers a topological perspective on relationships between concepts and applies it to practical concept interventions. The improvements in worst-group accuracy (WGA) on the erasure tasks are significant.
Critical weaknesses
- TCR does not show a clear advantage on the concept transfer task.
- TCR-DIS introduces a considerable preprocessing cost, according to Table 8.
Questions for authors
- How does the proposed topology-based approach differ theoretically from existing disentanglement methods, and what advantages does it offer? In Table 1, why does TCR-ERA outperform some TCR-DIS variants that explicitly model relationships between concepts?
- Could the authors evaluate TCR directly on established disentanglement benchmarks to clarify its advantage over existing methods?
5 comments
continuation of the previous comment
Moreover, many of these methods require enforcing factor-specific latent components and explicitly specifying their dimensions. In contrast, TCR-DIS does not prescribe how concept information should be distributed across the representation. Instead, it characterizes disentanglement through the interference , measuring how much erasing reduces the recoverability of . By reducing this interference while preserving individual concept recoverability, TCR-DIS directly targets our intended intervention behavior, without requiring statistical independence or a predefined factorization. We will add these clarifications in the related work of the revised manuscript.
Beyond this distinction in disentanglement objectives, TCR provides a general representation of concepts and their interactions. Rather than designing a dedicated formulation for each intervention, different objectives, including concept erasure, disentanglement, and transfer, can be expressed by specifying different target concept organizations within the same optimization framework. Persistence provides the underlying topological abstraction, characterizing how concepts connect across different interaction levels and identifying the relationships responsible for merging concept groups. These relationships can then be selectively targeted and updated as the concept organization evolves during optimization.
Regarding established disentanglement methods and benchmarks, we would be interested in better understanding which ones the reviewer was indeed thinking about. Could the reviewer clarify whether they have in mind classical unsupervised methods, such as β-VAE and FactorVAE, or supervised approaches, such as DIVA, CCVAE, and Guided-VAE? Similarly, are they referring to benchmarks such as dSprites, Shapes3D, MPI3D, or CelebA?
References
[1] Higgins et al. (2017). β-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. ICLR.
[2] Kim & Mnih (2018). Disentangling by Factorising. ICML.
[3] Chen et al. (2018). Isolating Sources of Disentanglement in Variational Autoencoders. NeurIPS.
[4] Kumar et al. (2018). Variational Inference of Disentangled Latent Concepts from Unlabeled Observations. ICLR.
[5] Ilse et al. (2020). DIVA: Domain Invariant Variational Autoencoders. MIDL.
[6] Joy et al. (2021). Capturing Label Characteristics in VAEs. ICLR.
[7] Siddharth et al. (2017). Learning Disentangled Representations with Semi-Supervised Deep Generative Models. NeurIPS.
[8] Klys et al. (2018). Learning Latent Subspaces in Variational Autoencoders. NeurIPS.
[9] Ridgeway & Mozer (2018). Learning Deep Disentangled Embeddings With the F-Statistic Loss. NeurIPS.
[10] Bousmalis et al. (2016). Domain Separation Networks. NeurIPS.
[11] Bertoin & Rachelson (2024). Disentanglement by Cyclic Reconstruction. IEEE TNNLS.
[12] Gabbay & Hoshen (2020). Demystifying Inter-Class Disentanglement. ICLR.
[13] Ding et al. (2020). Guided Variational Autoencoder for Disentanglement Learning. CVPR.
"In Table 1, why does TCR-ERA outperform some TCR-DIS variants that explicitly model relationships between concepts?"
TCR-ERA and TCR-DIS address related but distinct objectives, which may explain their differences in performance. Although TCR-ERA does not explicitly model concept interactions, it directly optimizes the removal of the unwanted concept while preserving task recoverability, allowing it to achieve a favorable trade-off between the two. In contrast, TCR-DIS first reduces interference between concepts while preserving their individual recoverability, before applying a separate erasure method. Its main benefit is therefore to provide a general preprocessing step that can facilitate subsequent erasure using different methods, rather than directly optimizing the final erasure outcome.
Importantly, TCR-ERA does not systematically outperform TCR-DIS. For instance, TCR-DIS combined with the best-performing erasure method achieves higher WGA on Waterbirds (74.66 vs. 71.17), DIAL-Mention (58.20 vs. 55.70) and Jigsaw (61.05 vs. 57.37), whereas TCR-ERA performs better on Bias in Bios (86.52 vs. 83.50). These results highlight the complementary roles of both approaches: TCR-ERA offers effective direct erasure, while TCR-DIS provides a reusable preprocessing step that can improve diverse existing erasure methods.
We hope that our responses and the additional evidence provided address your main concerns. We would be happy to clarify any remaining questions and welcome your reassessment in light of these clarifications.
Engaging with the authors
Dear reviewer,
the authors have posted a reply to your review. Please have a look and interact with the authors as needed.
Thank you, Your AC
We thank the reviewer for their comments and questions, which we address point by point below.
The improvements in concept transfer are indeed more moderate than those observed for concept erasure, particularly for ShuffleNetV2, where SPKD remains the strongest baseline. Nevertheless, the results provide encouraging evidence for the potential of TCR in this setting.
In particular, for ResNet8×4, TCR achieves the highest concept recoverability (60.05), improving by 5.54 points over the strongest baseline (SPKD, 54.51), and by 5.98 points over standard KD (54.07), while also achieving the highest top-1 accuracy (73.68). Beyond the concepts explicitly optimized, TCR also improves recoverability on independently discovered concept banks. For instance, at the finest spatial scale, optimizing 20 concepts improves recoverability on a separately discovered 10-concept bank by 4.94 points over SPKD (Fig. 4).
While these improvements do not establish a consistent advantage across architectures, we believe they demonstrate the promise of using a common concept-level abstraction for knowledge transfer, particularly for preserving information that conventional distillation objectives may overlook. We see this as an encouraging direction for further investigation.
TCR-DIS indeed introduces an additional preprocessing cost, although its absolute runtime remains relatively moderate in several settings. For instance, combining TCR-DIS with lightweight methods such as LEACE or SPLINCE requires only 33–89 seconds in total across the four datasets (Table 8), including both preprocessing and erasure. For comparison, standalone RLACE requires 39–209 seconds and LEOPARD 11–132 seconds, with TCR-DIS+LEACE/SPLINCE being faster than both on three of the four datasets. More computationally intensive combinations can nevertheless be substantially slower, notably TCR-DIS+RLACE on DIAL-Mention (1001 seconds, compared to 209 seconds for RLACE alone). Thus, while the additional cost is not negligible, combining TCR-DIS with efficient erasure methods can remain computationally competitive with existing approaches, while providing substantial improvements in task preservation.
A clearer comparison with existing disentanglement methods would indeed help position our contribution. Classical unsupervised methods, such as β-VAE [1], FactorVAE [2], β-TCVAE [3], and DIP-VAE [4], aim to discover separately encoded generative factors without supervision regarding which factors should be disentangled. Their objectives therefore differ substantially from our setting, where the concepts of interest are specified and the goal is to modify their relationships within a pretrained representation.
Although different, supervised disentanglement methods share more similarities, as they use labels to guide the separation of specified factors, that could be interpreted as the concepts of interest in our setting. These methods typically aim to learn representations with separated factors directly from the data, rather than reorganize concepts already encoded in a pretrained model. While they could be adapted to operate on pretrained features, this is not their primary setting. Methods such as DIVA [5], CCVAE [6], Siddharth et al. [7], CSVAE [8], F-Statistic Loss [9], and DSN [10] encourage, through different mechanisms, factor information to be represented in separate components (i.e. subspaces) of the optimized representation. However, encouraging each factor to be encoded in a dedicated subspace does not necessarily ensure its information from not being encoded in other subspaces. Consequently, erasing a concept in a subsequent step may still substantially affect the recoverability of another, whereas TCR-DIS directly optimizes this interference, being the targeted use-case in our work. Other methods, such as DiCyR [11] and LORD [12], isolate a single supervised factor from complementary information, but do not explicitly protect the recoverability of a second specified factor, which may consequently be damaged during disentanglement or subsequent erasure, contrary to TCR-DIS. Guided-VAE [13] goes further by adversarially discouraging factor information outside their designated components, but targets a broader statistical separation than required for selective erasure, potentially introducing unnecessary optimization constraints when applied in our use-case, whereas TCR-DIS directly targets erasure-induced interference.
continuation in the next comment