acceptodds
Under review as a conference paper at ICLR 2027

Global Concept Sparsity for Unsupervised Concept Bottleneck Models

Abstract

Concept bottleneck models (CBMs) aim to make predictions interpretable through human-understandable concepts, yet unsupervised approaches for automatically discovering concepts produce large, redundant concept vocabularies. While existing sparsity mechanisms constrain which concepts are active for particular inputs or classes, in practice they allow for redundant concepts which are used only for a small number of classes. As a result, locally sparse models may still rely on an effectively large global concept vocabulary, undermining the compactness and inspectability that concept bottlenecks are intended to provide. To address this, we propose Global Concept Gating (GCG), a global concept-gating mechanism that can be easily used within any unsupervised CBM. GCG learns a sparse mask shared across all inputs and classes, jointly selecting a compact task-specific concept set and training the downstream classifier. Across multiple concept bottleneck architectures and image classification tasks, GCG substantially reduces concept set size while largely preserving predictive accuracy. Further analysis shows that the retained concepts are more relevant to prediction, better semantically grounded, and less susceptible to information leakage. Our results highlight that interpretable concept-based models require not only meaningful individual concepts, but concept spaces appropriately constrained to their tasks.

Primary area:
interpretability and explainable AI
Keywords:
Concept Bottleneck Models, Explainable AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_AMJcOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper investigates global concept sparsity in unsupervised CBMs. The main motivation is that existing approaches use a large concept vocabulary while enforcing sparsity only at the input or class level, and reducing the size of the global vocabulary may improve interpretability and reduce information leakage. To do this, the paper proposes Global Concept Gating (GCG), a method that learns a task-specific subset of concepts via a sparse mask shared across all inputs and classes. The classifier is then retrained with the selected concepts, effectively reducing the global vocabulary while also preserving most of the classification accuracy.

Critical strengths

Reducing bottleneck size while keeping accuracy is a useful improvement over state-of-the-art CBMs, although a bit incremental in my opinion, especially for DN-CBM where oversizing the bottleneck is by-design to allow for task-agnosticism.

Critical weaknesses

The benefit of reducing the global concept vocabulary is not demonstrated. The main motivation and claim of the paper is that a large concept set is problematic for interpretability and information leakage, but there is not enough evidence to support this claim. Regarding interpretability, the main evidence is a few examples in Figures 5 and 6 where top concepts account for most of the prediction after GCG, however this depends on sparsity at the input and class level, rather than the vocabulary size. Other CBMs such as VLG-CBM and M-CBM already provide such concise explanations while maintaining larger concept vocabularies. Furthermore, the paper reports a small improvement in similarity with CUB attributes before and after applying GCG to DN-CBM and claims better semantic grounding, however DN-CBM deliberately learns task-agnostic concepts and therefore it is not particularly surprising that selecting a task specific subset results in higher alignment with concepts annotated for that dataset and this does not demonstrate that concepts are more semantically grounded or interpretable or easier to audit. Regarding information leakage, the paper compares selected concepts against random projections at different bottleneck sizes, but this only shows that the selected concepts are more predictive than random projections, not that leakage is reduced. A given concept bottleneck can beat random projections significantly but still have substantial leakage, as it was shown for VLG-CBM in the M-CBM paper. Additionally, all these experiements were done only against DN-CBM, which as I already mentioned, has a large bottleneck by design to make the bottleneck task-agnostic, a property that would naturally not be conserved after applying a task-specific selection.

Questions for authors

N/A

3 comments

Thank you for taking the time to go over this work and provide us very useful feedback. We are glad you found that our work provides a "useful improvement" for state-of-the-art CBMs. Your suggestions and comments were very helpful for us improving our manuscript, and we hope we can address your remaining concerns in this reply.

We reply to your concerns and questions below and have updated our paper accordingly, marking new changes in blue. If you have further concerns, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your review in light of our replies below.

[W1] The benefit of reducing the global concept vocabulary is not demonstrated.

Thank you for raising this. We agree that the benefit of a smaller vocabulary is ultimately a claim about human understanding, which we do not measure directly with a user study. However, there is well-established evidence that it matters, and we refer to these studies in our Introduction (L53-74). We also restate the studies here:

  • Working memory is limited. People can hold only a handful of items in mind at once (Miller, 1956; Cowan, 2001).
  • Interpretability studies find the same. Explanation complexity, such as the number of features or cognitive chunks, reduces how accurately and quickly users can simulate and verify a model (Lage et al., 2019). For concept-based explanations specifically, users struggle as the number of concepts grows (Ramaswamy et al., 2023).

Verifying a model as a whole means reviewing every concept it can use. Under these findings, a vocabulary of a few hundred concepts is far more feasible to review than one of 8192. Requiring a deployment study before accepting this would be like requiring that improved test accuracy be validated in deployment before it is taken as evidence. The proxy is standard because the link is well supported.

We have also added direct evidence that the smaller vocabulary is more useful in Section 4.6 and Appendix A.5:

  • On standard CBMs with annotated concepts, GCG reduces concept leakage (see W5).
  • On DN-CBM, the concepts GCG retains are task-relevant, whereas the most influential removed ones are largely unrelated to the task (new Figure 7).

[W2] Examples in Figures 5 and 6 depend on input-level sparsity rather than vocabulary size

Thank you for this observation. Our quantitative evaluation of effective concept size (Section 4.2) shows that the two are closely linked. To make this visible in the examples, we have updated Figure 5 to use absolute contributions, with "Other" as the summed magnitude of the remaining concepts, which avoids signed terms cancelling.

This shows that the baseline's explanation of a single image draws on a long tail of concepts from across its full vocabulary.

  • The tail is large in the baseline. For the CIFAR-100 example in Figure 5, the 8187 unshown baseline concepts carry a summed contribution of 15.5. For the GCG model, the 607 unshown concepts carry 9.5.
  • GCG concentrates the decision. Its top five concepts carry about twice the baseline's share of total contribution (18.6% vs 8.5%).

The per-input tail is therefore a consequence of the large vocabulary: a model that may use any of 8192 concepts spreads each decision across many of them. Reducing the vocabulary shortens that tail.

[W3] Other CBMs already provide sparse explanations while having large vocabularies

Thank you for raising this. We agree that per-input sparsity and vocabulary size are different properties, and we have attempted to explain this in the Introduction.

Our focus is global verifiability: being able to review every concept the model can use, across all inputs. For this, the effective vocabulary is what matters, not the per-input explanation size. A model can be sparse for each input yet draw on a different set of concepts each time.

This is what we observe in the existing sparse CBMs. In LF-CBM and UCBM, each prediction may come from a sparse set of concepts, yet the union of concepts used across the test set covers nearly the entire dictionary (Table 1).

GCG is complementary to input-level sparsity rather than a replacement for it. It bounds the set of concepts that any explanation can draw on, and it can be combined with the native sparsity mechanisms of these models.

[W4] Evidence for better semantic grounding

Thank you for this comment. We agree that our original grounding evaluation was coarse, because its concept-to-attribute mapping dropped body-part information (e.g. "has wing color::olive" became "olive").

In response, we have added experiments on standard (vanilla) CBMs trained on CUB's 312 annotated attributes (Section 4.6). There, every concept has a known meaning and a ground-truth label, so grounding can be measured directly.

  • Concept accuracy is preserved. In sequential and independent CBMs, the concept predictors are unaffected by the gate. In the joint CBM, GCG removes 73.7% of the concepts at a cost of only 1.8 points of concept accuracy (Table 3).
  • The concepts stay meaningful to the model. The gated models are also still responsive to concept interventions (Figure 9).

[W5] Evidence for reduction in leakage

In response, we have added a controlled test in which every concept has a known meaning. We trained standard CBMs (sequential, independent and joint) on CUB's annotated attributes, with and without GCG. We then measured concept–task leakage (CTL) and inter-concept leakage (ICL) following Parisini et al. (2025). These metrics quantify the information that predicted concepts carry about the task, or about each other, beyond what the ground-truth concept labels carry. Our new results, summarised in Section 4.6, suggest that both leakage metrics are reduced when GCG Is applied to the vanilla CBM. This shows that GCG can reduce leakage while maintaining a smaller vocabulary and without significant sacrifices to either task or concept accuracy.

[W6] Using DN-CBM as a benchmark

Thank you for this point. We chose DN-CBM because it has the largest concept vocabulary of the models we study: 8192 sparse-autoencoder concepts, learned without reference to the downstream task. This is precisely the setting in which an uncontrolled vocabulary is most problematic and in which global selection is most needed.

We acknowledge that DN-CBM's concept dictionary is trained on CC3M, which limits its accuracy on fine-grained datasets such as CUB (as its authors note). For this reason, our conclusions do not rest on DN-CBM alone.

  • We evaluate GCG on three further CBMs whose concepts are built for each dataset (LF-CBM, UCBM and VLG-CBM).
  • We have added experiments on standard CBMs trained on CUB's ground-truth attributes (see W4 and W5). These do not depend on DN-CBM's dictionary.

I thank the authors for the rebuttal and the additional experiments.

Regarding W1, if I understand correctly, the cited studies investigate cognitive limitations and the complexity of individual explanations, therefore it may hint at benefits for decision-level sparsity and explanation size but do not suggest benefits of global sparsity.

Regarding W2, I appreciate the correction to Figure 5, but this does not address my concern. A model can have highly concentrated contributions from a few concepts even with a large global vocabulary.

Regarding W3, I agree that global and decision-level sparsity are distinct, and I understand the motivation for improving global auditability, but the benefit of reducing the global vocabulary for auditability is still assumed rather than demonstrated.

Regarding W4, the new experiments on supervised CBMs show that concept accuracy is (mostly) preserved when applying GCG, but that does not demonstrate an improvement in semantic grounding.

Regarding W5, I appreciate the additional experiments on information leakage in supervised CBMs. However, the paper focuses on unsupervised CBMs, so it would maybe be more relevant to investigate the effect of GCG on information leakage in those models.

Regarding W6, as I mentioned in my initial review, the DN-CBM vocabulary is large by design to support reuse across tasks. Showing that it can be significantly reduced for a specific task remains somewhat trivial for DN-CBM specifically.

Overall, I appreciate the clarifications, but most of my concerns remain, and therefore I think my original score was adequate.

Reviewer_ZpCiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies global concept sparsity in unsupervised concept bottleneck models (CBMs). Its main point is that using few concepts for each image or class does not mean that the whole model uses a small concept set. The authors propose Global Concept Gating (GCG). It learns one concept mask shared by all inputs and classes. The concept encoder stays fixed. After learning the mask, the authors remove the closed concepts and refit the classifier on the retained set.

The paper evaluates four CBMs on CIFAR-100, CUB, and Places-365. It reports accuracy and global concept counts. For DN-CBM, it also compares GCG with random subsets and weight-level L1 sparsity. Further DN-CBM tests study attribute alignment on CUB and compare concept features with random projections. The results support the value of global concept selection. The claims about lower information leakage and better auditability need stronger evidence.

Critical strengths
  1. The paper identifies a useful gap in how concept sparsity is measured. Table 1 shows that 11 of the 12 settings use at least 62% of their concept vocabulary across the test set. This makes the motivation concrete. The authors also explain that this test-set union can still miss concepts used on other inputs. This finding supports reporting global concept use alongside per-image or per-class sparsity.

  2. The compression results cover several models and tasks. Table 2 includes four CBMs and three datasets. For DN-CBM on CUB, the effective concept count falls from 5,138 to 242.2, while accuracy rises from 67.72% to 68.18%. On Places-365, the count falls from 8,192 to 1,694.8, while accuracy rises from 47.65% to 52.90%. The paper includes five final-head training seeds, random subsets of the same size, and accuracy-versus-budget curves. The mask-and-refit step also makes the final concept count meaningful. Together, these results show practical value.

Critical weaknesses
  1. The experiments do not yet isolate the value of GCG from general global selection and refitting. The paper itself frames the problem as global feature selection. However, the main controls are random subsets and the original weight-level L1 probe. A random subset is useful, but it is a weak test of a task-trained selector. Weight-level L1 also does not directly target a shared set of concept columns across classes.

    There is a second issue in Figure 3. GCG selects a set and then refits the classifier. It is not clear whether the L1 baseline gets the same refit after fixing its selected set. Without this control, some of the gap could come from less weight shrinkage or the extra training stage. I am not assuming that the authors omitted this step, but the protocol needs to be stated.

    The most useful fix is a comparison on the same frozen features, with the same global concept budget, validation rule, and refit procedure. It should include L1 selection followed by refitting, and a method that directly selects shared concept columns. A group penalty on the classifier's concept columns is one simple option. An existing gate-based feature selector, such as the cited Stochastic Gates method, would also be relevant. Accuracy-versus-concept-count curves would show whether GCG adds value beyond these choices.

  2. The evidence for lower leakage and better auditability is weaker than the paper claims. The CUB grounding result is a positive finding. Jaccard improves from 0.2695 to 0.2982, with a reported bootstrap 95% confidence interval of [0.024, 0.033] for the paired gain. The same-size random subset scores 0.0216. However, the mapping removes body-part information, such as changing has wing color::olive to olive. It therefore tests coarse attribute overlap. It does not show correct location, reliable concept interventions, or easier model auditing.

    Figure 4 does not directly measure information beyond a concept's meaning. Learned concept features can beat random projections because they retain useful task information more efficiently. This can happen even if the retained activations still contain information outside their named concepts. Likewise, a high-dimensional random projection matching the model does not establish leakage in that model. The current test supports a result about feature compression, rather than a direct conclusion about semantic leakage.

    Figure 5 has a related limit. A small signed sum for the unshown concepts can result from positive and negative terms cancelling out. It does not show that these concepts have little total influence. Also, the predicted-class logit alone does not show their effect on the margin over other classes. Reporting absolute contributions and class-margin contributions would make this example more convincing.

    To keep the stronger claims, the authors should add a test with known concept meanings and controlled extra factors, or a carefully designed concept intervention test. The test should ask whether predictions rely on information outside those meanings. Otherwise, the abstract, contribution list, and conclusion should limit the claims to a smaller concept set and better average attribute alignment. A user study is not required to establish the compression result, but smaller vocabularies alone do not establish better audit performance.

Questions for authors
  1. In Figure 3, is the L1-selected concept set fixed and then refit with the same procedure as GCG? If so, please give the settings. If not, please add this control and at least one direct global selection baseline under matched concept budgets. This is my highest-priority request.

  2. Appendix A.3 says that training uses hard gates but evaluation of the intermediate model uses soft gates. Why is this change made? What is its accuracy with the hard mask used directly? Please report hard-mask, soft-gate, and refit results separately. Similar accuracy before and after refitting is also not enough to show similar behavior. Prediction agreement would be a more direct check.

  3. Is there evidence that separates reduced semantic leakage from better task-related compression? A controlled semantic test could change my view of the third contribution. If such evidence is not available, would the authors narrow the leakage and auditability claims?

2 comments

Thank you for carefully reviewing our manuscript and for your feedback. Your suggestions and comments were very helpful for us in improving our manuscript. We are glad you found that our work "identifies a useful gap" and that you found our evaluation extensive.

We address your concerns and questions below and have updated our paper accordingly, marking new changes in blue. If you have further concerns, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your review considering our rebuttal below.

[W1 and Q1] Isolating the value of GCG from general global selection and refitting.

Thank you so much for pointing this out. Initially, the L1 baseline in Figure 3 was not refitted, as we wished to compare the original sparsity mechanism of the baseline model directly to GCG (so we did not modify how they encouraged sparsity in the original baselines). Nevertheless, we acknowledge that adding a refitted baseline is a fair control for disentangling how much of our gains come from GCG’s selection and how much comes from refitting.

In response, we have updated our manuscript to include additional plots showing the performance of the baseline model with L1 sparsity and refitting in Section 4.4 (“L1 w/ refit” baseline in Figure 3). Our results, summarised in Figure 3, suggest that, although refitting restores much of the performance of the baseline L1 sparsity, this control still underperforms compared with GCG, especially at the operating percentage of concepts that we report.

[W2.1] CUB concept mapping removes body-part information.

For those experiments, we removed body-part information during evaluation because of the limited vocabulary of DN-CBM, which retains general characteristics of an image, but discards most location-specific traits. To evaluate GCG's influence on concept accuracy while controlling for the limited vocabulary of the model, we now introduce a new set experiments evaluating the concept accuracy of a vanilla CBM with GCG compared to baseline using CUB's ground truth concept annotations (Section 4.6). There, our results suggest that GCG decreases the model's concept accuracy slightly, despite reducing inter-concept and concept-task leakage.

[W2.2 and Q3] Figure 4 shows concept compression, and does not accurately measure information leakage.

We agree that Figure 4 alone may better represent GCG's compression abilities than semantic leakage. To separate the two, we ran a controlled experiment evaluating GCG on vanilla CBMs trained on CUB's ground-truth attributes (using the expert-annotated attributes in CUB as concepts). This lets us quantitatively measure leakage using concept-task leakage (CTL) and inter-concept leakage (ICL) metrics [1], which quantify the information that predicted concepts carry about the tasks and each other. Our new results, summarised in Section 4.6, suggest that both leakage metrics are reduced when GCG Is applied to the vanilla CBM. This shows that GCG can reduce leakage while maintaining a smaller vocabulary and without significant sacrifices to either task or concept accuracy.

[W2.3] Absolute contribution is not presented in Figure 5.

Thank you for pointing this out. We now provide the absolute contributions for the example in Figure 5. We find that the contribution of "other" concepts increases in both the baseline and the DN-CBM explanations. Still, the GCG model concentrates about twice the share of total contribution in its top five concepts and draws on 612 concepts rather than 8192. We have revised the text and Figure 5 to show this.

[Q2] Use of hard vs soft gates for evaluation

Thank you for catching this. The description of the evaluation in Appendix A.3 was incorrect. The gated model is trained and evaluated with the hard mask, and we have rewritten A.3 accordingly and re-reported the gated accuracies under this evaluation (Table 7). As requested, we also report hard-mask and refit accuracy separately for every setting in Appendix A.3. We also agree that similar accuracy does not imply similar behaviour, so we have added prediction agreement between the hard-masked and refit models. On DN-CBM, the refit (final GCG) model matches the hard-masked model on 97.0% (CIFAR-100), 97.9% (CUB), and 95.2% (Places365) of test images. However, agreement is lower for LF-CBM and UCBM on Places-365 (66% and 64%), where refitting re-optimises a probe that co-adapted to the mask during joint training. Both models use only the concepts in K, so the concept-set guarantee holds for either; we report both and have revised A.3 to no longer claim that refitting preserves behaviour in all settings.


References

  1. Parisini et al. "Leakage and interpretability in concept-based models." JMLR (2026).

Thank you for the clarifications and additional experiments. I appreciate the L1-with-refit comparison, the annotated-CUB leakage experiments, and the new per-class and per-image analysis. These additions address several concerns in my initial review.

My main concern remains the value of GCG beyond existing global selection methods. My original request included both L1 selection followed by refitting and a method that directly selects shared concept columns, evaluated on the same frozen features with matched concept budgets and refitting protocols. The added L1 comparison addresses the refitting confound, but the response does not report the latter comparison. This leaves the additional benefit of GCG’s selection mechanism insufficiently isolated.

The annotated-CUB results strengthen the leakage evidence, although their implications for the semantic fidelity and auditability of the unsupervised concept dictionaries remain limited. I recognize the practical value of the compression results, but the unresolved methodological comparison remains important to my assessment. I am therefore maintaining my rating.

Reviewer_XhWZOpenReview ↗
Rating
4(+2)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses the problem of building compact concept bottleneck models (CBMs) that are interpretable and inspectable. It notes that existing CBMs, even when they include sparsity objectives to ensure decisions use a small number of concepts, only do so at the sample or class level. As a result, the global set of concepts used by the model remains large. The paper then proposes global concept gating (GCG), a simple wrapper to learn a mask over the concept set when training for a task. This mask is then used as a hard threshold, which forces only a small subset of concepts to remain, and the final classifier is retrained. Across experiments over various CBMs, the paper shows that GCG yields CBMs that use significantly fewer concepts, have comparable accuracy as the originals, have significantly better accuracy when evaluated at the same global sparsity levels, and can help select more semantic concepts.

Critical strengths
  1. The paper addresses a useful problem that seems to have been surprisingly overlooked, to my knowledge, and the proposed approach is also simple but seems to work. Several approaches to sparsity have been proposed in prior work, but they all in one way or the other make the weights between the concepts and the output sparse, not the concept set itself. While methods like DN-CBM propose task-agnostic concept bottlenecks that are useful to prevent retraining for each task, it makes sense to subselect concepts after the bottleneck is trained, so the task remains interpretable.
  2. Quantitative evaluation is generally thorough and well thought out.

Other Strengths

  1. The writing is clear and easy to follow. The introduction shows a clear and compelling story.
Critical weaknesses
  1. My biggest concern is that almost the entire evaluation is quantitative, and gains are shown via various metrics. While this is of course important to ensure rigour and necessary to evaluate at scale, it is also important to actually look at the data. For example, Table 2 shows significantly fewer concepts being used. Which are the concepts that got removed? Are they truly redundant, or did we lose something in the long tail, that could, e.g., hurt robustness? It is plausible that there are several concepts that don't globally affect task performance, but are useful for rare instances of the task. A closer analysis on this would be helpful. I also feel there should be a lot more qualitative examples shown. Figure 5 and 6 only show top concepts for some examples, but there is nothing that shows in a global sense what was kept and what was removed. Another cause of concern is that many of the quantitative analyses rely on DN-CBM with CUB, but Appendix A of the DN-CBM paper states that performance on CUB is poor as the concept bottleneck is trained on CC3M. There is also no user study to evaluate concepts, which is a useful way of evaluating interpretability. Evaluations looking into this would be helpful.
  2. The main motivation of this work is using sparsity that helps understanding model behaviour as whole. However, this has not been discussed in the experiments and results. Since the bottleneck is now very sparse, it would be helpful to have concrete examples or case studies on what holistic understanding can be obtained, which was not possible before. This would significantly strengthen the paper.

Other Weaknesses

  1. In Section 4.5, an experiment is performed to evaluate how well concepts selected align with human expectations, and this is claimed to be a proxy for auditability. In my view the two are quite different. For example, if the model learnt to use poorly aligned concepts for a task, then those should be kept for auditing, instead of trying to align to what humans expect the models to learn. Similarly, information leakage experiments are purely done based on concept counts, but since there are ground truth annotations it could be useful to evaluate how well the concepts ground to the input via attribution methods.
  2. The paper combines class and input level sparsity, but they are different ideas. While it does not directly affect the core arguments, it would help if they were properly distinguished in the writing and discussion.

Overall, I have a positive view of the paper, but my rating is affected by the importance of these issues. I would be happy to update my score if a satisfactory response to the weaknesses and questions are provided in the rebuttal.

Questions for authors
  1. How is λgate\lambda_{\text{gate}} selected?
  2. How sensitive is the method to gate initialization? What if we initialize multiple times with gate values sampled from e.g. a gaussian (or another) distribution? The ablations in A.1 only seem to cover various constants. Or is there a strong reason for constant initialization?
  3. Why is there a discrepancy in concept counts for the baseline between Table 1 and Table 2? For example, for LF-CBM on places, Table 1 reports 1952/2161 while Table 2 reports 2145. Where is this difference coming from? L289-290 suggests it might be the number of concepts with non-zero weights, but if so, how is there a std of $\pm$23 for the baseline?
  4. What is the baseline value in Figure 3? This could be clearer.
5 comments

Thank you for taking the time to review our submission carefully. Your feedback was very helpful in improving our manuscript, and we are glad you found that our work addresses a “useful” and “overlooked” problem, that our quantitative evaluation is “thorough”, and that our paper’s writing is “clear”.

We address your concerns and questions below and have updated our paper accordingly, marking new changes in blue. If you have further concerns, please let us know. Otherwise, we would sincerely appreciate it if you would consider updating your review considering our rebuttal below.

[W1.1 and W2] Qualitative results showing the impact of our masking

We agree that qualitative studies are important. Nevertheless, in our evaluation we focused primarily on quantitative studies, as they allow us to evaluate our main hypotheses more objectively through standard scientific practices focused on properties we argue are important for CBM users (e.g., task accuracy, effective dictionary sizes, explanation accuracy, leakage, etc.). We acknowledge that our qualitative experiments are limited compared with our quantitative results, but larger qualitative analyses (e.g., user studies) are notoriously difficult to evaluate when ground truths are unavailable. Specifically, user studies bring all sorts of challenges and research questions that, in light of the large empirical and quantitative evaluation we already present in this work, we believe are best suited for a follow-up paper studying the actual auditability of CBMs (something that has not been done in our work or in any other previous work). With this in mind, we also highlight this as a limitation and suggest it as future work in Section 5.

That said, we agree it is helpful to analyse which concepts were selected and which were dropped. To help with this, we provide (1) a new set of experiments using GCG when we have ground-truth concepts in CUB (Section 4.6), and (2) a new appendix in Appendix A.5 showing the top retained and the highest-contribution removed concepts. Our results on CUB suggest that GCG is indeed capturing redundant concepts, as we see that for jointly trained CBMs on this version of CUB we can drop up to 73.7%73.7\% of the concepts. In contrast, task and concept accuracy decrease by only 0.81%0.81\% and 1.8%1.8\%, respectively. Complementing these results, our analysis of which concepts were dropped on CUB suggest that GCG is capable of retaining relevant concepts while discarding others for a given dowsntream task. We thank the reviewer for raising this point, and hope these experiments address their concern regarding our evaluation of the selected concept set.

[W1.2] Quantitative Evaluation of GCG on DN-CBM

As argued at the start of Section 4.4, we chose to conduct some quantitative analysis on DN-CBM because it “discovers a large concept vocabulary from a task-independent corpus” and therefore “provides a good basis for understanding how GCG selects task-relevant concepts”. Regarding the concern for DN-CBM’s performance on CUB, we note that the only DN-CBM results in the main body of our paper that are evaluated uniquely on CUB, and therefore not substantiated with similar trends on other datasets, are our experiments validating the semantic grounding (Section 4.5). All other experiments show similar trends for DN-CBMs in multiple datasets, including CUB. Therefore, our overall experiments show that our results are not specific to DN-CBMs applied only to CUB. In fact, as shown in our new results in Section 4.6, we observe similar trends when applying GCG to different CBM architectures trained using a different concept set in CUB.

[W3] Auditability and leakage evidence

We updated the first paragraph in Section 4.5 to clarify that semantic groundness can support auditability (even if it does not guarantee it or it is equivalent to it).

Regarding your comments on leakage, we added a new section (Section 4.6) that explores leakage in a setting with ground-truth concept labels, namely CUB with expert-derived attributes as concepts. That enables us to measure it using CTL and ICL [2]. Our results show that concept sets selected by GCG reduce leakage without significantly impacting the models’ task and concept accuracies.

References

  1. Zhao et al. "Partially shared concept bottleneck models." AAAI (2026).
  2. Parisini et al. "Leakage and interpretability in concept-based models." JMLR (2026).

[Q1] How is λgate\lambda_\text{gate} chosen?

We choose λgate\lambda_\text{gate} by grid searching over eight values in [10−5,10−2][10^{-5}, 10^{-2}], with β=0.25\beta = 0.25, and selecting the value that maximises validation accuracy and validation CEA. This approach is similar to that used in Zhao et al. [1]. We specify this in Section 4.3.

[Q2] Sensitivity to initialization

Thank you for bringing forth this question. We opted for a constant initialisation for (1) robustness, as we want our method to be robust to random seeds, (2) symmetry, as we do not want to impose a prior that prioritises certain concepts for selection over others, and (3) simplicity, as this results in having a single hyperparameter. For this reason, we set all gate logits to +2 (σ(2)≈0.88\sigma(2) ≈ 0.88) to ensure every concept starts open and enters training on equal terms, so selection depends only on the data and the sparsity penalty.

In contrast, under random initialisation, a gate whose initial logit is negative starts closed: its concept is masked in the forward pass, the classifier learns no weight for it, and it can reopen only through the straight-through gradient. To put this in context, if one initialises the masks with N(2,σ2)\mathcal{N}(2, \sigma^2), the share of gates that start closed is about 0% for σ=0.5\sigma = 0.5, 2.3% for σ=1\sigma = 1, and 15.9% for σ=2\sigma = 2.

Nevertheless, as suggested, we tested random initialisation and include some new results on this ablation in Appendix A.1. We drew the gate logits from Gaussians N(2,σ2)\mathcal{N}(2, \sigma^2), and from a uniform U(1,3)U(1, 3), with five independent draws per setting, and kept the rest of the training recipe unchanged. We found that random initialisation has little effect on the accuracy/sparsity produced by GCG, but it leads to greater inconsistencies in the global concept set the model retains across runs.

A random initialisation changes this. A gate whose initial logit is negative starts closed: its concept is masked in the forward pass, the classifier learns no weight for it, and it can only reopen through the straight-through gradient. Under N(2, σ2\sigma^2), the share of gates that start closed is about 0% for σ=0.5\sigma = 0.5, 2.3% for σ=1\sigma = 1 and 15.9% for σ=2\sigma = 2.

As suggested, we tested random initialization. We drew the gate logits from Gaussians N(2, σ2\sigma^2) and from a uniform U(1, 3), with five independent draws per setting, and kept the rest of the training recipe unchanged. We found that random initialization has little effect on the accuracy/sparsity produced by GCG, but it does lead to greater inconsistencies in the global concept set retained by the model across runs. We add these results in Section A.1.

[Q3] Apparent discrepancy in concept counts for the baseline between Table 1 and Table 2

Thank you for raising this. The two tables use different definitions. Specifically:

  • Table 1 counts concepts with a non-zero contribution ∣Wy^,jc^j(x)∣|W_{\hat{y},j} \hat{c}_j(x)| to the predicted class on at least one test image.
  • Table 2 counts concepts with at least one non-zero weight in the final layer, which bounds concept use over all inputs.

Thank you for spotting this. The discrepancy arose because some Table 1 counts were computed from different probes than those in Table 2. We have recomputed Table 1 on the same probes used for Table 2, so both tables now describe the same models; The conclusions of Section 4.2 are unchanged. We have clarified both definitions in the captions.

[Q4] Baseline value in Figure 3

Thank you for pointing this out - the baseline value in Figure 3 represents the task accuracy of the baseline model (i.e., the model without any masking). We have clarified this in the figure caption.

Thank you for the reply and the new results. Q1-Q4 is well addressed, and I appreciate the toning down in W3. For W1, by qualitative I did not mean user study, but in the sense of qualitative examples (such as in Figure 6). Looking at the data is almost always helpful and would also help readers, so I strongly encourage adding more. The results in Appendix A.5 are helpful, but it would be great to have this analyses over all models and datasets, as this is not something that is expensive to obtain and requires no extra training.

The results in the new Section 4.6 are great to have, and I like the fact that it separates out GCG performance from that of the concept extraction method. However, here as well it would be helpful to have a more qualitative understanding beyond numbers, and not in the aggregate sense. For example, for the same class, how do the predictions before and after GCG differ? Are there a specific type of images from a class that are hurt by concept removal, e.g., a view of the class that is a minority? Aggregate analyses are helpful but also a bit coarse.

It also seems my W2 and W4 have not been addressed, and I would encourage the authors to do so when revising.

Overall, I am borderline positive about this paper, and given the additional analyses that partly address my concerns, I am increasing my score to a 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.