acceptodds
Withdrawn from ICLR 2027

Intervening in Concept Bottleneck Model via Causal Effects from Pixel to Concept

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

15 views

Abstract

Concept bottleneck models (CBMs) employ human-interpretable concepts for predictions, enabling corrective interventions in neural networks. However, current intervention methods are usually limited to the concept bottlenecks encoded by features or their corresponding weight matrices, leading to inefficient interventions, limited interpretability, and dependence on human priors that restrict generalization. To address this, we propose CBM-PCCEI, a Pixel-to-Concept Causal Effect Intervention method for CBMs. It utilizes causal effects hierarchically to achieve multi-level intervention, improves effectiveness, and provides a more easily understood explanation of reasoning and intervention. To generalize improvements beyond the intervened cases and reduce dependence on human priors, we integrate the interventions with knowledge distillation, transferring the improved behavior to the base model. Evaluations on Flower-102, Aircraft, CUB-200, CIFAR-10, and CIFAR-100 with NFResNet-50 and DaViT show that our method improves validation accuracy by up to 11.01% with an average gain of 6.62% via single patch intervention, and achieves up to 3.82% with an average of 1.48% testing accuracy gain under generalization settings, outperforming both vanilla fine-tuning and state-of-the-art intervention strategies.

Primary area:
interpretability and explainable AI
Keywords:
Concept Bottleneck Model, Explainable Artificial Intelligence, Neural Network Intervention

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_ccYdOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This work introduces CBM-PCCEI, a framework for automated interventions on concept bottleneck models. This method features interventions in three different fashions: At the concept level, at the image-patch level, and combined hierarchical concept-to-patch intervention. All interventions are performed by perturbing parts of the input (or concept) space randomly and finding perturbations that improve the output logits of the true class. Based on these interventions, this work further introduces a training setup that utilizes automated interventions on the validation set to finetune the model and improve its performance without any test-time interventions.

Critical strengths
  • The general motivation of the paper to identify systematic biases in the concept extractor and correct them is important, and current CBM intervention approaches are not able to do this (they can at most put a band-aid over the problem, but not solve it from the ground up).
Critical weaknesses
  • The intervention procedure of the paper is introduced as an automatic intervention mechanism that does not require human input or prior knowledge. However, the method requires arguably even more critical information to be used, namely the label of the true class for a sample at test time. This is a fact which makes the method impractical for test-time intervention purposes, and this fact is not mentioned at all in the paper.
  • The paper is lacking in readability and clarity. The motivation and structure of the written text are often unclear, mixing different lines of argument and making it hard to follow the text. The formal notation of the method is inconsistent, often lacking clear and proper explanations, and goes against typical conventions for mathematical notation, which makes it even harder to read.
Questions for authors

Major Points:

Major discussion points of the intervention method:

  1. The proposed intervention method is motivated as not requiring human input and prior knowledge. However, the method requires knowledge of the true class label to make any of its interventions, which is never mentioned explicitly in the paper. In my opinion, this makes the method completely impractical as a test-time intervention method in terms of altering the downstream prediction on a per-sample basis. If the correct label is already known, there is no big benefit in going through lots of computations to perturb the input/concepts so that the model predicts the correct class better.
  2. The main and potentially interesting merit of the method is to use such automated interventions in advance (i.e. in a finetuning phase during model development) or in an online learning setting at test time to improve the performance of the model down the line.
  3. When comparing to classical intervention approaches, the difference in available information is not discussed at all, making all comparisons highly biased towards the method introduced in this paper, as stronger information is available.
  4. In terms of motivating the method as an intervention method in the typical setting of CBMs, it should also be noted that the assumptions and procedure are completely different from what intervention methods typically are considered. In typical CBM literature, interventions are considered correcting (or at least trying to correct) the concept predictions of the model. In this paper, the proposed method instead perturbs parts of the concept space/input space to random values, which alters the input to the model and does not consider the correctness of the perturbations in any form.

Due to these points, I think this work needs a major revision. However, I am happy to see the authors discussing these points in the rebuttal.

Formal introduction of the method:

The formal introduction of the method has a lot of inconsistencies and could be written more clearly.

  • For clarity, it would help to follow typical conventions for notation. For example, vectors are usually marked as lowercase bold x\mathbf{x}. This work mixes scalars, vectors, function symbols, and sets, and is further inconsistent in how the individual elements are represented throughout the paper.
  • Furthermore, the terminology in the paper is not really clear and varies often in terms of how parts of the model and mathematical aspects are named. It would help to follow typical conventions of how elements are named in the field, e.g. see [1] for an overview.
  • What exactly is vcptv_\text{cpt}? As far as I understand it, this is just the weight matrix that maps from the latent embedding of the feature extractor to the concept activations. However, why is qq then defined as a function of viv_i and vcptv_\text{cpt} ? In typical conventions, the parameters of a model are not written as a value that a function gets, but maybe as an index (which you also do for WW and hh in l. 176). This notation is rather confusing and inconsistent.
  • l. 170: Reuse the definitions for the number of classes and concepts introduced in 3.1
  • Eq. 3: kk was previously introduced as the dimensionality of the latent embedding of the model, and is now reused here with a different meaning
  • Eq.4: What is nfinen_\text{fine}? and how are W1,…,Wncoarse\mathcal{W}_1, \dots, \mathcal{W}_{n_\text{coarse}} defined? Are they partially the same as the coarse ones, or new independently drawn values? is nfinen_\text{fine} larger or smaller than ncoarsen_\text{coarse}?
  • l. 225: vv is reused, it was the latent embedding of the feature extractor before and is now a patch index (which gets even more confusing in the next paragraph on hierarchical interventions).
  • l. 257: It is confusing that c^\hat{c} is defined as a function, while cc is only the concept representation, indices could be better suited here.
  • l. 259: Inconsistencies in the usage of c−c⁻ and c+c⁺ . Are these both the two parts of c^\hat{c} ? In the next equations, the symbols are already used differently.
  • l. 262: It looks like there is a part of the equation missing, should the c^−\hat{c}^- part not also be relative to the non-intervened concept values?
  • It might be a bit confusing that Mcpt\mathcal{M}_\text{cpt} and Mcls\mathcal{M}_\text{cls} operate on different things. The class metric operates on the absolute difference of margins, while the concept-level one operates on the difference of concept activation norms (averaged over all concepts at the same time).
  • explanations via concept causal attributions:
    • In this section, it is really hard to follow the text. New elements are repeatedly mentioned, but not properly introduced, and the notation is not consistent between the algorithm and the text (e.g. what G_c is)
    • Can you briefly explain how the class-specific crops in Dcrops\mathcal{D}_\text{crops} are obtained?
    • What is p^\hat{p} ?
    • Can you properly introduce pfeap^\text{fea} - across what interventions is this computed?
    • what is Mc−vcptM_c - v_\text{cpt} ?
    • where do the natural language concepts with largest GcG_c come from? (in particular, the language part)?
    • consequently, what is the text that is embedded by the text encoder of CLIP?
    • l. 311: the dataset introduction is inconsistent with the initial data introduction in 3.1

Story and red line:

In the following, I group smaller points regarding the story of the paper and the line of arguments:

  • l. 41: This work claims that CBMs are more difficult to comprehend compared to other conceptual explanations and attribution techniques, however this claim is not supported with any references or further discussed. Can the authors please discuss this claim? (in particular in the context that there is a lot of literature showing that pixel-based explanations are difficult to interpret: [2, 3])
  • l. 161: There is a lot of work using CLIP-based concepts, these works should be discussed a bit better (see. e.g. [1] for an overview)
  • l. 182: Why does beginning with interventions on negative concepts balance interpretability with intervention effectiveness?
  • l. 226: Why should a crop from DIV2K be in the distribution of the data that is used for the model? The input images could for example have a different style, or be from a different domain
  • Explanations via concept causal attribution:
    • As far as I understand, this is just providing visual explanations for concepts, and there are works that already do this (e.g. [4, 5]), how does this differ from these methods?
  • It would be really helpful to separate the experimental setup, dataset and model information into an explicit part/subsection at the beginning of the experiments to improve the readability of the section.
  • Tab. 1:
    • It is difficult to follow what the evaluation should actually achieve, because it is not aligned with some underlying hypothesis or research question that should be verified.
    • What is the standard deviation of the base model (before interventions) over multiple seeds?

Further discussion points:

Comparison against CBM-HNMU:

  • I think the comparison against this method should be motivated more clearly and explained better in the main paper: CBM-HNMU is designed to improve a blackbox model and uses a CBM as a surrogate, which differs quite a bit from the setting of this work. Can the authors explain what the setting for CBM-HNMU in each of the experiments is and how it is compared to this method?

Using automated interventions as a distillation target for the model:

  • Can the authors discuss a bit more why they think the distillation of the automated intervention logits is helpful, and maybe connect this to the initial motivation of reducing systematic biases of the concept extractor?
  • Can the authors discuss why there are differences between the distillation and no-distillation settings regarding P\mathcal{P} ? ("inherent limitations of the distillation process" is a bit vague). Furthermore, how do the results in Tab. 3 look for the other datasets?

Effectiveness of the resulting attributions:

  • In the example in Fig. 3, the causal effect map (which is the bases for the explanation), basically only highlights the blossoms of the sample image. However, when looking at the explanations for the purple and yellow concepts, the examples highlight a lot of green, leafy patches. Furthermore, there is a high overlap between the example images of the different concepts, which does not seem helpful. Can the authors explain why this is the case?

User Study:

  • For an intervention method given this setting (not the typical CBM interventions), it would be best to intervene (perturb) concepts that are cues of the other classes, not of the ground truth class. However, the method shows an improvement on CT and CR cues, which the method should perturb. Can the authors please discuss why higher values for CT and CR are desired?

Minor things:

  • l. 48: I think this should be hallucinations instead of illusion
  • l.200: "disturbance concept" is an unusual naming, maybe "the perturbed concept" would be better
  • l.328: Add references for the backbone models in the text as well?

[1] Knab, Patrick, et al. "What’s in the bottle? a survey and roadmap of concept bottleneck models." Transactions on Machine Learning Research (2026). [2] Kim, Been, et al. "Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav)." International conference on machine learning. PMLR, 2018. [3] Adebayo, Julius, et al. "Sanity checks for saliency maps." Advances in neural information processing systems 31 (2018). [4] Fel, Thomas, et al. "Craft: Concept recursive activation factorization for explainability." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023. [5] Benou, Itay, and Tammy Riklin Raviv. "Show and tell: Visually explainable deep neural nets via spatially-aware concept bottleneck models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

Reviewer_DERtOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents CBM-PCCEI, a method for intervening in concept bottleneck models. It first uses random changes to concept-to-class weights to select harmful concepts. It then replaces image patches to reduce these concepts while keeping other concept scores close to their original values. The corrected outputs and concept scores are used to train the original model through distillation. This allows prediction on new images without labels or direct intervention at test time.

The paper evaluates the method on five datasets with NFResNet-50 and DaViT. It includes comparisons with fine-tuning and other intervention methods, component studies, runtime results, and a user study. For NFResNet-50, the reported average accuracy gains are 6.62 percentage points for direct intervention on validation images and 1.48 points on the test set after distillation.

Critical strengths

1. The intervention can act on the input features through explicit concept goals.

Editing the concept layer alone cannot change what the feature extractor sees. The proposed method gives a way to connect a concept correction to an image region. Keeping other concept scores stable is a sensible goal. If the selection rule is reliable, this could make input edits more targeted and easier to explain.

2. The experiments provide real evidence of transfer beyond the edited samples.

The paper tests two backbones on five datasets. Section 3.4 uses the same selected samples and optimization settings for distillation and fine-tuning. This is an important control. In Table 2, HI-Gen improves over matched fine-tuning by about 0.64 percentage points on average for NFResNet-50 and 0.62 for DaViT.

The appendix also includes CI/PI/HI comparisons, gradient replacements, grayscale masks versus DIV2K crops, removal of CLIP supervision, and a matched comparison with CBM-HNMU. These studies make the empirical case stronger. The paper is not relying only on a few successful examples.

Critical weaknesses
  1. The concept-selection score does not match the interventions subsequently performed.

    This is my main concern. Under the linear classifier specified in Secs. 3.1–3.2, changing only a concept's target-class weight leaves all competing class logits fixed. The resulting margin change is therefore exactly the concept activation multiplied by the change in that weight. However, the selection procedure evaluates random weight replacements, while the actual concept intervention on page 5 sets the selected weight to zero. These operations can produce effects with opposite signs.

    For example, suppose the concept activation is 1 and its target-class weight is 0.25. If replacement values are sampled uniformly between 0 and 1, their mean is 0.5, giving an expected margin improvement of 0.25. Setting the original weight to zero instead reduces the margin by 0.25. Thus, the selection rule can assign a positive score to a weight whose deletion worsens the classification margin.

    The transition to HI introduces a further mismatch. Changing a target-class weight affects only that class, whereas reducing a concept activation through an image edit affects every class. For a fixed competing class, the effect on the margin depends on the difference between the concept's weights for the two classes. A concept that contributes negatively to the true-class logit may contribute even more negatively to a competing class, meaning that it supports discrimination between them. Suppressing that concept can therefore reduce the margin even when deleting its target-class weight would improve it. The paper does not establish why concepts selected through weight perturbation should be suppressed through the subsequent pixel intervention.

    Please clarify whether the implementation follows the stated equations and explain how the selection criterion corresponds to the intervention actually performed. A targeted comparison with a criterion that directly evaluates the executed intervention, under otherwise matched settings, would help assess the practical impact of this mismatch. Reporting how often the selection score agrees in sign with the observed intervention gain would also be informative. Furthermore, the expected weight-replacement score can be calculated analytically under the stated linear classifier, so the need for repeated sampling at this stage requires explanation.

    The reported HI improvements may still be valid, but the current formulation does not justify the claimed harmful-concept selection mechanism. Since the selected concept set also guides patch selection and explanation construction, this concern affects a central component of the method.

  2. The explanation and bias-correction claims are not sufficiently supported by the current evaluation.

    The user study in Sec. 4.5 and Appendix E evaluates samples that were successfully corrected by the intervention. Its scores assess concept presence and perceived confusability in the input and examples of the confused class. Concepts that participants cannot associate with visual features are skipped and excluded from evaluation. Participants may also assess relational phrases primarily through individual attributes, such as judging "surrounded by orange" mainly through the presence of "orange."

    These results provide evidence of visual relevance. They do not establish that the full concept meaning is faithfully represented, that the attributed image regions correspond to that meaning, or that the stated concept explains why the prediction changed. These distinctions matter because the paper claims to explain both the model's reasoning and the intervention mechanism. The concept-shift evaluation in Appendix D does not resolve this concern: cosine distance measures changes in vector direction and is insensitive to positive rescaling, so it cannot establish that the magnitudes of harmful concept scores were reduced.

    Please report the proportion of concepts excluded from the user study and the agreement between participants. A targeted assessment that also includes failed interventions and initially correct predictions would help clarify the scope of the explanation claims. More directly, the evaluation should test whether modifying the concept named in an explanation changes the classification margin in the predicted direction, while accounting for changes in other concepts.

    The claim of correcting upstream feature bias also requires more direct evidence or narrower wording. An accuracy improvement after editing an image does not by itself establish that a spurious feature dependency has been identified and removed. The current results support useful correction on the evaluated datasets, but do not yet establish a general mechanism for removing upstream bias. The introduction and conclusions should reflect this evidential scope.

Questions for authors
  1. Does the implementation use exactly the linear classifier and the single-entry weight change described in Equations (1)-(5)? What is the sampling distribution on [0,1][0,1]? If the implementation differs, please give the actual computation and explain whether it changes the sign-mismatch example above.

  2. What are the exact dimensions of the patch features, cluster centers, and concept embeddings in Algorithm 1? What mapping puts them in a shared space before the distance is computed?

  3. Since the validation data are used for distillation, what data are used for early stopping, loss-weight selection, and other model choices? Please give the split sizes. What sources of randomness are included in the reported error bars?

  4. Is the HI header in Table X meant to be PI? What is the total time for concept selection, patch scoring, final intervention, and distillation under the reported setup?

Reviewer_tL9vOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes CBM-PCCEL that extends interventions in a CBM model to multiple levels: concepts, pixels, and a hierarchical one. Image level features are mapped first to concepts by a network, which are mapped to final labels by another one. For concept level interventions, an expected gain metric MclsM_{cls} measures the change in softmax margin due to a concept being manipulated for each concept. Similarly, a metric Δpatch\Delta_{patch} is (persumably on the soft max for concepts) calculated for pixel patch level intervention. A metric McptM_{cpt} combines them for hierarchical interventions. Losses derived from these metrics form the final intervention. Results on different CBM methods for a few datasets are shown to study the impact of pixel level, concept level, and hierarchical interventions. A user study is also presented.

Critical strengths

The analysis using margins of the softmax layer at the concept and output layers and metrics derived from there seem interesting (though I found them very difficult to comprehend).

Results and ablations in the main paper and in the ultra-long appendix seem to be comprehensive.

Critical weaknesses

Comprehensibility of the method is very low and that has seriously hampered by understanding. This his not so much about the language but about the explanations that are assumed, which do not agree with my evaluation. Here are a few points to illustrate this:

  • L203. In the interval [0,1] or one of 0 or 1? Earlier discussion indicated the latter. Not clear
  • How is ncoarsen_{coarse} selected? And the corresponding sets?
  • What is the intuition behind the vv values? Average shift in margin when concept ii is replaced with a set of constant values. What does it indicate? Why is it important?
  • “approximating the average treatment effect of concept interventions” is written above. But why these specific interventions involving WW values?
  • What are the coarse and fine values? What are we trying to zoom into? How are they different? How do we set threshold θ1\theta_1? How is nfinen_{fine} selected? What is the intuition?
  • Line 218: Selected N concepts with maximum margin. Set them to 0. How does that reduce negative conceptual influence and correcet prediction?
  • Are vv values sorted in ascending or descending order? I would think the bottom ones will have negative influence and top ones will have positive influence. What am I missing?
  • I have the same question on the pixel level interventions also. What are the coarse/fine numbers? What is the intuition? If you sort mm values in descending order, won't the top ones have positive influence?
  • L247: I don't understand how it "removes harmful pixels". Please explain.
  • Again, I am confused about the c+c^+ and c−c^- used in L262 and around there.
  • Algorithm 1 talks about IexpI_{exp} , IcropsI^{crops}, etc. Where do these come from? I don't see the exp or crops explained in this context earlier. The algorithm's connection with prior description is difficult to figure out.
  • I also don't see a clear description of the hierarchical intervention algorithm.
Questions for authors

My main issues are the comprehensibility and utility/intuition behind the method. That makes it impossible for me to validate the method and hence the results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.