acceptodds
Under review as a conference paper at ICLR 2027

A Unifying Framework of Concept-based Explainable AI with Completeness Guarantees

Abstract

Concept-based explanations describe neural network predictions through human-understandable properties of inputs called concepts. The field encompasses approaches that differ in how they define and represent concepts and connect them to model predictions. We introduce a theoretical framework that describes these approaches in a common mathematical language and supports a shared analysis of their properties. For concept discovery, which identifies concepts automatically within a latent space of a trained model, we employ a concept autoencoder view. An encoder extracts concept representations from the model's latent space, and a decoder uses them to reconstruct the original latent representation. The autoencoder’s reconstruction error measures how accurately its decoder recovers the original latent representation. We revisit model completeness: how well the concepts can reproduce the model's outputs. We show that model incompleteness of the concepts can be bounded by the autoencoder's reconstruction error. The autoencoder view also provides a common way to define individual concept attributions, which measure each concept's contribution to a prediction. We establish when these attributions sum to the model's prediction, and bound the discrepancy otherwise, thus providing attribution completeness guarantees.

Primary area:
interpretability and explainable AI
Keywords:
explainable AI, concepts, concept-based XAI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UgVFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a common mathematical framework for concept-based explanations, distinguishing input-level concepts, concept representations, and scalar activations. It describes concept discovery through an encoder–decoder pair that reconstructs a model’s latent representation. The main results connect reconstruction error to prediction fidelity and model completeness, and separate attribution error into fidelity and additivity errors. The paper establishes exact additivity for affine decoded prediction heads and a curvature-based bound for smooth nonlinear heads. Experiments with ResNet-50 on ImageNet examine these relationships across five concept extraction methods.

Critical strengths
  1. The framework makes useful distinctions between different notions of completeness. Sections 2–4 distinguish the information retained by concept representations, the fidelity of a particular reconstructed predictor, and whether concept attributions sum to its output. In particular, the separation between fidelity and additivity clarifies why an attribution can perfectly explain the reconstructed predictor while still disagreeing with the original model. This is a useful perspective for interpreting existing concept-based explanations.

  2. The formulation accommodates different concept representations within a consistent analysis. Separating multidimensional representations from scalar activations allows the framework to describe bottleneck models, embedding models, and several discovery methods. The worked examples in Section 3 make these connections concrete. Section 4 also states explicit conditions under which reconstruction and attribution become exact, allowing the reader to identify which guarantees apply to a particular construction.

Critical weaknesses
  1. The main theoretical contribution remains modest relative to existing unifying frameworks. Fel et al. (NeurIPS 2023) already connect dictionary-based concept extraction with attribution methods, while Poché et al. (ACL 2025) explicitly formulate concept encoders, decoders, and reconstructed predictors. MCD (TMLR 2023) also provides concept-level completeness through multidimensional decompositions. Section 5 acknowledges these connections, but the additional results in Section 4 largely follow from standard properties: Lipschitz continuity in Theorem 1, the definition of an infimum in Theorem 2, the triangle inequality in Eq. (13), and affine decomposition in Proposition 1. The broader notation and separation of errors are useful, but the paper does not sufficiently demonstrate what substantial new conclusion about concept-based explanations follows from them. This is particularly important because the shared formulation and guarantees constitute the paper’s stated contribution.

  2. The main evaluation does not establish that the guarantees are informative beyond the affine setting. Section 4.4 evaluates two ResNet-50 boundaries where the prediction head is affine. With affine decoders, the experiments confirm the exact additivity already implied by Proposition 1. The nonlinear decoder provides the main test of the broader attribution guarantee, but its bound exceeds the measured attribution error by two to three orders of magnitude. The fidelity bound is also approximately four to ten times the measured error. These findings characterize the conservativeness of the bounds, but do not demonstrate when the guarantees can meaningfully certify explanation accuracy or help choose between concept representations and attribution rules. Since the paper’s central value rests on its completeness guarantees, an informative application of those guarantees would materially strengthen the contribution.

[1] Fel, T., Boutin, V., Béthune, L., Cadène, R., Moayeri, M., Andéol, L., ... & Serre, T. (2023). A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems, 36, 54805-54818.

[2] Poché, A., Jacovi, A., Picard, A. M., Boutin, V., & Jourdan, F. (2025, July). Consim: Measuring concept-based explanations’ effectiveness with automated simulatability. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 5594-5615).

[3] Vielhaben, J., Bluecher, S., & Strodthoff, N. (2023). Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees. Transactions on Machine Learning Research (TMLR), 2023.

Questions for authors
  1. Which concrete conclusion about existing explanation methods becomes available through the proposed framework beyond the encoder–decoder formulations and completeness results discussed in Section 5? A worked example where the new analysis changes how a method is assessed would help clarify the contribution.

  2. Among the settings evaluated in Section 4.4, is there a case where the bounds provide an informative certificate or distinguish between methods in a practically meaningful way? In particular, how should the nonlinear attribution bound be used given its reported gap of two to three orders of magnitude?

  3. How does the attribution analysis relate to methods that already satisfy completeness for nonlinear predictors? For example, Integrated Gradients, aggregated over concept blocks, can exactly recover the decoded predictor’s output difference under its standard assumptions (Sundararajan et al., ICML 2017). Clarifying why the proposed analysis focuses on the three selected rules would help establish its scope and added value.

[1] Sundararajan, M., Taly, A., & Yan, Q. (2017, July). Axiomatic attribution for deep networks. In International conference on machine learning (pp. 3319-3328). PMLR.

1 comment
Area_Chair_UirXOpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Reviewer_P4dAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper looks at a comprehensive theoretical analysis of the underpinnings of concept-based models. They instantiate several popular concept-based models within this framework, such as CBMs, TCAVs, and CEMs. They then prove bounds on different types of error (e.g., reconstruction errors), and show how these errors are related to each other. They conclude by establishing the tightness of their error bounds in practice on a variety of models.

Critical strengths
  1. Adds much needed theory to concept-based models - The paper takes a stab at formalizing some of the work in concept-based models. In general, much of the work in concept-based models is empirical, and so adding a theoretical component is helpful and important. It puts the entire field on a firmer basis, and also allows us to compare between different models and settings.
Critical weaknesses
  1. Unclear what the main message is - From reading the paper, it's unclear to me what the main takeaway should be. The bounds are nice for the comparison between different forms of error, but it doesn't seem to enlighten much in terms of how different models work or their relationship. I saw this paper as analogous to a literature review in some sense, and as a result, what might have instead been useful is essentially pointing out what major gaps remain in concept-based models, what should be explored, etc. The paper as it's written doesn't hint at this, and so makes it hard to see where to go next.
  2. Difficult to read through dense math in general - The paper presents a variety of bounds, but it's hard to interpret the practical meaning of these bounds .This becomes especially difficult due in part to the large number of functions being used, making it hard to distinguish what different error terms mean. More broadly, the presentation largely states theorems and bounds, without relaying their practical significance.
Questions for authors
  1. What exactly is the practical takeaway from this work?
2 comments
Area_Chair_UirXOpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Reviewer_P4dAOpenReview ↗

Rebuttal Reply

Thank you for the rebuttal, and for the changes made to the manuscript. I think I just want a bit more concreteness on what each of your points mean here. In particular:

  1. "Makes existing methods comparable" - What does this mean exactly? Are existing methods not already comparable empirically? Is the primary difference between many common methods (e.g., CBM and CEM) their representation of the middle concept layer?
  2. "Identifies which conclusions transfer between them" - What does a conclusion mean? Are you saying what properties of CBMs transfer over to CEMs or TCAVs for example? What kinds of properties could we convincingly show this for theoretically (I get the bounds on the loss) that would improve our intuition beyond empirical experiments? Or is it that this would allow us to have more information before running any experiment?
  3. "The practical takeaway is to distinguish information retained by concept representations" - I appreciate this paragraph, but I'm still a bit confused. How exactly would you probe which of these issues it is, and what kind of experiments would you need for that?

I appreciate the changes that are being made by the authors as well; I think those changes can help make the paper clearer and easier to follow, with the main message better made upfront.

Reviewer_QjRjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a unifying framework for concept-based XAI, casting concept discovery methods as concept autoencoders on a model's latent space. Its main results are bounds showing that, for a Lipschitz prediction head, fidelity error and model completeness error are controlled by reconstruction error, and that attribution error is bounded by fidelity error plus an additivity error. Concept attribution methods are analysed. Experiments on ResNet-50 confirm that all bounds hold but are often loose.

Critical strengths
  1. The paper is clear and easy to follow. The mathematics is explained well.
  2. The framework seems sensible, and as far as I can tell all the results presented in the paper are correct.
Critical weaknesses
  1. I did not find many of the results in the paper surprising, and the utility of the framework is not completely clear to me.
  2. The framework does not attempt to model whether concepts are useful or interpretable. This is understandable, as modelling these properties is very difficult, but it is a limitation of the framework: perfect reconstruction error could be obtained with opaque representations. Furthermore, the work does not consider concept interventions (where incorrect concept predictions are corrected) or concept leakage (where information unrelated to a concept is contained in its prediction or embedding).

# Minor Suggestions

These minor issues did not affect my rating and do not need to be addressed in the rebuttal (although they could be fixed in a future iteration of the manuscript).

  • Figure 2 appears after Figure 3 in the manuscript.
  • Some of the detail in the Introduction could be moved out of the Introduction and into later sections.
  • References to previous work should be cited (e.g., on lines 167 and 172).
Questions for authors
  1. What are the concrete benefits of the framework? Can it uncover any surprising results? What potential impact could it have?
  2. Would it be possible to incorporate concept interventions or concept leakage into the framework?
4 comments
Area_Chair_UirXOpenReview ↗

Engaging with the authors

Dear reviewer,

the authors have posted a reply to your review. Please have a look and interact with the authors as needed.

Thank you, Your AC

Thank you for your response.

Significance of the framework

I'm afraid that I am still not convinced that the framework has led to substantial new insights into concept-based methods, and it isn't clear to me what the main, novel takeaway of the work is.

Incorporating concept interventions

I do not believe that this intervention formulation captures how CEMs are intervened on. A CEM keeps the input's own embeddings c^i+(x),c^i−(x)\hat{c}_i^+(x), \hat{c}_i^-(x) and sets the mixing probability to the true label ci∈{0,1}c_i \in \{0,1\}, so no correction policy is needed.

Because my concerns about the significance/utility of the framework remain, I will leave my score unchanged.

We thank you for your feedback on our response. We would like to ask what you would consider a substantial contribution from an abstract framework.

As it stands, our framework allows a range of XAI methods to be formulated abstractly using a unified notation. This facilitates the transfer of results and ideas from one method to another, as illustrated, for example, by the result on attribution completeness.

We would greatly appreciate your perspective on which additional properties, results, or applications would most strengthen the contribution of the proposed framework.

Thank you for the question. By a substantial contribution I mean that the framework leads to something that would be hard to get without it. Unified notation is useful, but on its own it is mainly a matter of presentation.

Any one of the following would considerably strengthen the paper:

  1. A non-trivial transferred result. A guarantee or insight established for one family of methods that, through the framework, gives a new result for another family, one that was not known and does not follow directly from the definitions.
  2. A practical consequence. A new concept-discovery method, attribution rule or evaluation protocol derived from the framework that measurably improves on existing ones.
  3. Evidence that the distinctions matter in practice. Experiments across several architectures and datasets showing, for example, that separating fidelity from additivity changes which attribution methods look best, or explains failures that attribution error alone hides.

I hope this is helpful, and I appreciate the authors' engagement.

Comments on the paper

1 comment

Thank you for the constructive feedback. We have revised the manuscript to emphasize the framework as the central contribution and its guarantees as reusable consequences.

  • QjRj — usefulness, interventions, and leakage: Section 3.3 now formalizes interventions on concept representations and states when prediction can depend entirely on scalar concept activations. We clarify how this relates to information beyond those activations, while distinguishing it from semantic leakage. Section 4.3 explicitly expresses insertion and occlusion as interventions.

  • P4dA — main message and practical interpretation: The introduction and conclusion now foreground the common formulation and its use across methods. Sections 4.2–4.3 add concrete interpretations and examples: complete representations can coexist with a deficient decoder, and changing between the three attribution rules cannot improve output recovery for a fixed affine decoded head.

  • UgVF — added value, nonlinear guarantees, and Integrated Gradients: Section 5 clarifies the relationship to existing frameworks. Section 4.3 and Appendix B explain how Integrated Gradients fits the analysis and add a smooth nonlinear example with a known curvature constant. Section 4.4 and Appendix C clarify that the ReLU experiment and sampled gradient ratios do not certify the smooth curvature bound, and that fitted heads only upper-bound model completeness error.

We also improved the exposition and figure placement. The original framework definitions in Section 2 and all reported experimental values remain unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.