acceptodds
Under review as a conference paper at ICLR 2027

Concept-Based Local Unified Explanations

Abstract

There is an increasing need to integrate model-agnostic explanation techniques with concept-based approaches, as the former can explain models across different architectures while the latter makes explanations more faithful and understandable to end-users. However, existing concept-based model-agnostic explanation methods are limited in scope, mainly focusing on attribution-based explanations while neglecting diverse forms like sufficient conditions and counterfactuals, thus narrowing their utility. To bridge this gap, we propose a general framework ConLUX to elevate existing local model-agnostic techniques to provide concept-based explanations. Our key insight is that we can uniformly extend existing local model-agnostic methods to provide unified concept-based explanations with large pre-trained model perturbation. We have instantiated ConLUX to provide concept-based explanations in three forms: attributions, sufficient conditions, and counterfactuals, and applied it to popular text, image, and multimodal models. Our evaluation results demonstrate that ConLUX provides explanations more faithful than state-of-the-art concept-based explanation methods, and provides richer explanation forms that satisfy various user needs.

Primary area:
interpretability and explainable AI
Keywords:
XAI, concept-based, model-agnostic

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3GqYOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to improve existing model-agnostic XAI methods by introducing concept-level inputs. In particular, unlike earlier work, e.g., LIME, that rely on unsupervised methods to extract text corpus or visually coherent visual patches for downstream operations, the proposed method leverages LLMs or SAM segmentation models to localize words and object masks for diverse concepts. The resulting inputs are then fed to the original XAI methods to generate the corresponding explanations. Experimental results show that the method is able to achieve favorable improvement on several concept-related metrics.

Critical strengths

(1) It is an important problem to enhance attribution-based XAI methods by incorporating more interpretable concepts.

(2) The proposed method is model-agnostic and does not rely on additional annotations.

(3) The paper considers a collection of tasks for evaluation, which is useful for achieving a comprehensive understanding of the method.

Critical weaknesses

(1) While the paper repeatedly emphasizes the need of concept-level perturbation for existing XAI methods, it is generally not a new idea to incorporate concept-based features/inputs with model-agnostic methods (e.g., ConceptLIME already explores the integration). The proposed method essentially replaces feature extraction modules in prior methods with an LLM or the SAM segmentation model, and has rather limited technical novelty.

(2) While Table 2 and 3 show considerable improvements, all of the baselines are relatively old and rely on weaker feature extraction approaches. As a result, it is not surprising that using more advanced LLM/Segmentation would result in higher accuracy.

(3) Related to the previous comment. The paper compares with other concept-based methods in Table 4. Nevertheless, these methods typically rely on different forms of inputs for perturbation. How do the authors ensure a fair comparison with consistent inputs? Section 4.2.2 states that concept-level neighborhood is used, but its technical definition is unclear.

(4) The evaluation metrics (e.g., accuracy) examines whether the specific concepts are removed or detected after diverse perturbation methods. It does not verify whether the perturbation affects the other concepts within the original scenes. The method uses a diffusion model for perturbation, and might introduce/remove other concepts.

(5) The human evaluation appears to be insufficient to me. In particular, it only takes into account 10 samples per task (5 for testing). Only two models are compared, and the test images are generated with concept-level perturbation that could be more favorable for the proposed method.

Questions for authors

(1) Please elaborate on the concept-level and feature-level neighborhoods, and how they are used during the evaluation.

(2) What are the key advantages of the proposed method over recent ones that also consider concept extraction?

Reviewer_PMkwOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a simple technique to "elevate" explanation methods from feature-based to concept-based. That way, explanations on the level of concepts can be obtained for black-box models using existing explanation algorithms. The algorithm is 1) extract concepts from the input; 2) perturb the concepts; 3) use (V)LM to generate synthetic examples from the perturbed concepts. Explanations are then learned on this. The contribution lies in the generality of the approach: many explanation algorithms across many tasks could potentially be elevated to provide concept-level explanations. The evaluation shows positive trend on a somewhat limited number of tasks, even though I do have some questions on the evaluation side of things.

Critical strengths
  • The method is simple and general, potentially providing useful, concept-level explanations. It seems clear, that such concept-level explanations would be more useful for the users.

  • The method is applicable to many different base algorithms, and to black-box models.

Critical weaknesses
  • The paper is rather hard to follow and repetitive in the background and methodology part (pages 1-5). The method itself is rather simple and in my opinion does not need the heavy treatment, which hinders, not helps the understanding. It can be explained in relatively small space, but the explanation should be better supported by examples, which are presently insufficient. On the other hand, the evaluation is underdeveloped. The evaluation methodology description leaves a number of questions in my mind and the method itself is evaluated on a small number of very simple tasks. For a method where such generality is claimed, I would expect a considerably more thorough evaluation in terms of the spectrum of task types.

  • I have reservations about some aspects of the evaluation, which I will detail in the questions for authors. Most notably, since the baselines (feature-level) and the method (concept-level) work in very different levels of granularity, I suspect that one of the central metrics biases towards the more coarse level of the proposed method (see questions below).

Questions for authors

L238: I am left wondering how the method ensures that when b_i^c=0 the sample will not satisfy the concept i. Is that explicitly enforced in the generation? I studied the prompt A.2 but do not see there any such instruction.

L297-303: "concepts include entity types such as..." leaves genuinely ambiguous whether the concepts are the entities themselves or their types...? Is the concept-to-feature mapping asked to remove all PERSON tags in the text, or to remove some particular person? In general, the description of the evaluation protocol is much too short for proper understanding.

L311: Does accuracy here account also for the absence? I.e. if the perturbation asks for the presence of a particular person and absence of a particular place, then will the accuracy account for that? Or does it only measure presence?

Table 3: Perhaps my most important reservation: Comparison of the feature-based method to its elevated variant runs on different scale. Removing one concept is a small change for the metrics in the concept space, but may correspond to a disproportionally larger change in the text and thus have a better chance at changing the prediction. Same does not hold for tokens. In my view, Table 3 should be adjusted for deletion budget, i.e. effect in the underlying text.

Isn't there tension in the locality of the perturbations. In the method proposed in the paper, the concept-feature map generates a wholly new text that has the correct mix of concepts. I think that is not a local change.

L456: Why would coverage be a metric which should be maximized. If coverage measures the proportion of perturbed examples where the model keeps its prediction, then I should think coverage depends on the perturbations and not the explanation. Why is coverage then presented as a metric to maximize?

Reviewer_T8wwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes to incorporate concept based perturbations into explainability pipelines involving LIME, Kernel SHAP, Anchors, and LORE using a combination of concept extraction techniques (segmentation for images and LLMs for text) and generative models for concept-feature mapping. The framework is evaluated against the four base techniques along with relevant concept based explanation techniques.

Critical strengths

The framework is simple and easy to fit in existing pipelines across image, and text modalities without requiring any model internals or gradient access. The use of concepts for rule based explanations in this context is interesting and under explored.

Critical weaknesses

1)The two main components in the framework which sets it apart from the base explainability techniques (four methods) are using concept extraction techniques and using generative models for perturbation. These have been already explored in the prior work in these settings. [2,3] uses generative models for perturbation in explainability techniques while augmenting LIME, SHAP, and Anchors have already been explored in [1].

2)No component ablations and missing preservation check: The paper doesnt show how the two new components (concept extraction and generative based perturbation) added contribute to the gains produced by the framework. Additionally, there are no experiments to evaluate if the perturbation has preserved unrelated information and locality ( the input is not changed in unintended ways). This draws questions on faithfulness of explanations provided by the framework.

[1] Liu, Junhao, and Xin Zhang. "ReX: A framework for incorporating temporal information in model-agnostic local explanation techniques." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 18. 2025.

[2] Chang, Chun-Hao, et al. "Explaining image classifiers by counterfactual generation." arXiv preprint arXiv:1807.08024 (2018).

[3] Agarwal, Chirag, and Anh Nguyen. "Explaining image classifiers by removing input features using generative models." Asian Conference on Computer Vision. Cham: Springer International Publishing, 2020.

Questions for authors

1)Can you add missing component ablations?

2)Can you provide evaluation on how the generative perturbation affects the unrelated parts of the input?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.