- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This work introduces CBM-PCCEI, a framework for automated interventions on concept bottleneck models. This method features interventions in three different fashions: At the concept level, at the image-patch level, and combined hierarchical concept-to-patch intervention. All interventions are performed by perturbing parts of the input (or concept) space randomly and finding perturbations that improve the output logits of the true class. Based on these interventions, this work further introduces a training setup that utilizes automated interventions on the validation set to finetune the model and improve its performance without any test-time interventions.
Critical strengths
- The general motivation of the paper to identify systematic biases in the concept extractor and correct them is important, and current CBM intervention approaches are not able to do this (they can at most put a band-aid over the problem, but not solve it from the ground up).
Critical weaknesses
- The intervention procedure of the paper is introduced as an automatic intervention mechanism that does not require human input or prior knowledge. However, the method requires arguably even more critical information to be used, namely the label of the true class for a sample at test time. This is a fact which makes the method impractical for test-time intervention purposes, and this fact is not mentioned at all in the paper.
- The paper is lacking in readability and clarity. The motivation and structure of the written text are often unclear, mixing different lines of argument and making it hard to follow the text. The formal notation of the method is inconsistent, often lacking clear and proper explanations, and goes against typical conventions for mathematical notation, which makes it even harder to read.
Questions for authors
Major Points:
Major discussion points of the intervention method:
- The proposed intervention method is motivated as not requiring human input and prior knowledge. However, the method requires knowledge of the true class label to make any of its interventions, which is never mentioned explicitly in the paper. In my opinion, this makes the method completely impractical as a test-time intervention method in terms of altering the downstream prediction on a per-sample basis. If the correct label is already known, there is no big benefit in going through lots of computations to perturb the input/concepts so that the model predicts the correct class better.
- The main and potentially interesting merit of the method is to use such automated interventions in advance (i.e. in a finetuning phase during model development) or in an online learning setting at test time to improve the performance of the model down the line.
- When comparing to classical intervention approaches, the difference in available information is not discussed at all, making all comparisons highly biased towards the method introduced in this paper, as stronger information is available.
- In terms of motivating the method as an intervention method in the typical setting of CBMs, it should also be noted that the assumptions and procedure are completely different from what intervention methods typically are considered. In typical CBM literature, interventions are considered correcting (or at least trying to correct) the concept predictions of the model. In this paper, the proposed method instead perturbs parts of the concept space/input space to random values, which alters the input to the model and does not consider the correctness of the perturbations in any form.
Due to these points, I think this work needs a major revision. However, I am happy to see the authors discussing these points in the rebuttal.
Formal introduction of the method:
The formal introduction of the method has a lot of inconsistencies and could be written more clearly.
- For clarity, it would help to follow typical conventions for notation. For example, vectors are usually marked as lowercase bold . This work mixes scalars, vectors, function symbols, and sets, and is further inconsistent in how the individual elements are represented throughout the paper.
- Furthermore, the terminology in the paper is not really clear and varies often in terms of how parts of the model and mathematical aspects are named. It would help to follow typical conventions of how elements are named in the field, e.g. see [1] for an overview.
- What exactly is ? As far as I understand it, this is just the weight matrix that maps from the latent embedding of the feature extractor to the concept activations. However, why is then defined as a function of and ? In typical conventions, the parameters of a model are not written as a value that a function gets, but maybe as an index (which you also do for and in l. 176). This notation is rather confusing and inconsistent.
- l. 170: Reuse the definitions for the number of classes and concepts introduced in 3.1
- Eq. 3: was previously introduced as the dimensionality of the latent embedding of the model, and is now reused here with a different meaning
- Eq.4: What is ? and how are defined? Are they partially the same as the coarse ones, or new independently drawn values? is larger or smaller than ?
- l. 225: is reused, it was the latent embedding of the feature extractor before and is now a patch index (which gets even more confusing in the next paragraph on hierarchical interventions).
- l. 257: It is confusing that is defined as a function, while is only the concept representation, indices could be better suited here.
- l. 259: Inconsistencies in the usage of and . Are these both the two parts of ? In the next equations, the symbols are already used differently.
- l. 262: It looks like there is a part of the equation missing, should the part not also be relative to the non-intervened concept values?
- It might be a bit confusing that and operate on different things. The class metric operates on the absolute difference of margins, while the concept-level one operates on the difference of concept activation norms (averaged over all concepts at the same time).
- explanations via concept causal attributions:
- In this section, it is really hard to follow the text. New elements are repeatedly mentioned, but not properly introduced, and the notation is not consistent between the algorithm and the text (e.g. what G_c is)
- Can you briefly explain how the class-specific crops in are obtained?
- What is ?
- Can you properly introduce - across what interventions is this computed?
- what is ?
- where do the natural language concepts with largest come from? (in particular, the language part)?
- consequently, what is the text that is embedded by the text encoder of CLIP?
- l. 311: the dataset introduction is inconsistent with the initial data introduction in 3.1
Story and red line:
In the following, I group smaller points regarding the story of the paper and the line of arguments:
- l. 41: This work claims that CBMs are more difficult to comprehend compared to other conceptual explanations and attribution techniques, however this claim is not supported with any references or further discussed. Can the authors please discuss this claim? (in particular in the context that there is a lot of literature showing that pixel-based explanations are difficult to interpret: [2, 3])
- l. 161: There is a lot of work using CLIP-based concepts, these works should be discussed a bit better (see. e.g. [1] for an overview)
- l. 182: Why does beginning with interventions on negative concepts balance interpretability with intervention effectiveness?
- l. 226: Why should a crop from DIV2K be in the distribution of the data that is used for the model? The input images could for example have a different style, or be from a different domain
- Explanations via concept causal attribution:
- As far as I understand, this is just providing visual explanations for concepts, and there are works that already do this (e.g. [4, 5]), how does this differ from these methods?
- It would be really helpful to separate the experimental setup, dataset and model information into an explicit part/subsection at the beginning of the experiments to improve the readability of the section.
- Tab. 1:
- It is difficult to follow what the evaluation should actually achieve, because it is not aligned with some underlying hypothesis or research question that should be verified.
- What is the standard deviation of the base model (before interventions) over multiple seeds?
Further discussion points:
Comparison against CBM-HNMU:
- I think the comparison against this method should be motivated more clearly and explained better in the main paper: CBM-HNMU is designed to improve a blackbox model and uses a CBM as a surrogate, which differs quite a bit from the setting of this work. Can the authors explain what the setting for CBM-HNMU in each of the experiments is and how it is compared to this method?
Using automated interventions as a distillation target for the model:
- Can the authors discuss a bit more why they think the distillation of the automated intervention logits is helpful, and maybe connect this to the initial motivation of reducing systematic biases of the concept extractor?
- Can the authors discuss why there are differences between the distillation and no-distillation settings regarding ? ("inherent limitations of the distillation process" is a bit vague). Furthermore, how do the results in Tab. 3 look for the other datasets?
Effectiveness of the resulting attributions:
- In the example in Fig. 3, the causal effect map (which is the bases for the explanation), basically only highlights the blossoms of the sample image. However, when looking at the explanations for the purple and yellow concepts, the examples highlight a lot of green, leafy patches. Furthermore, there is a high overlap between the example images of the different concepts, which does not seem helpful. Can the authors explain why this is the case?
User Study:
- For an intervention method given this setting (not the typical CBM interventions), it would be best to intervene (perturb) concepts that are cues of the other classes, not of the ground truth class. However, the method shows an improvement on CT and CR cues, which the method should perturb. Can the authors please discuss why higher values for CT and CR are desired?
Minor things:
- l. 48: I think this should be hallucinations instead of illusion
- l.200: "disturbance concept" is an unusual naming, maybe "the perturbed concept" would be better
- l.328: Add references for the backbone models in the text as well?
[1] Knab, Patrick, et al. "What’s in the bottle? a survey and roadmap of concept bottleneck models." Transactions on Machine Learning Research (2026). [2] Kim, Been, et al. "Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)." International conference on machine learning. PMLR, 2018. [3] Adebayo, Julius, et al. "Sanity checks for saliency maps." Advances in neural information processing systems 31 (2018). [4] Fel, Thomas, et al. "Craft: Concept recursive activation factorization for explainability." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023. [5] Benou, Itay, and Tammy Riklin Raviv. "Show and tell: Visually explainable deep neural nets via spatially-aware concept bottleneck models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.