- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a common mathematical framework for concept-based explanations, distinguishing input-level concepts, concept representations, and scalar activations. It describes concept discovery through an encoder–decoder pair that reconstructs a model’s latent representation. The main results connect reconstruction error to prediction fidelity and model completeness, and separate attribution error into fidelity and additivity errors. The paper establishes exact additivity for affine decoded prediction heads and a curvature-based bound for smooth nonlinear heads. Experiments with ResNet-50 on ImageNet examine these relationships across five concept extraction methods.
Critical strengths
-
The framework makes useful distinctions between different notions of completeness. Sections 2–4 distinguish the information retained by concept representations, the fidelity of a particular reconstructed predictor, and whether concept attributions sum to its output. In particular, the separation between fidelity and additivity clarifies why an attribution can perfectly explain the reconstructed predictor while still disagreeing with the original model. This is a useful perspective for interpreting existing concept-based explanations.
-
The formulation accommodates different concept representations within a consistent analysis. Separating multidimensional representations from scalar activations allows the framework to describe bottleneck models, embedding models, and several discovery methods. The worked examples in Section 3 make these connections concrete. Section 4 also states explicit conditions under which reconstruction and attribution become exact, allowing the reader to identify which guarantees apply to a particular construction.
Critical weaknesses
-
The main theoretical contribution remains modest relative to existing unifying frameworks. Fel et al. (NeurIPS 2023) already connect dictionary-based concept extraction with attribution methods, while Poché et al. (ACL 2025) explicitly formulate concept encoders, decoders, and reconstructed predictors. MCD (TMLR 2023) also provides concept-level completeness through multidimensional decompositions. Section 5 acknowledges these connections, but the additional results in Section 4 largely follow from standard properties: Lipschitz continuity in Theorem 1, the definition of an infimum in Theorem 2, the triangle inequality in Eq. (13), and affine decomposition in Proposition 1. The broader notation and separation of errors are useful, but the paper does not sufficiently demonstrate what substantial new conclusion about concept-based explanations follows from them. This is particularly important because the shared formulation and guarantees constitute the paper’s stated contribution.
-
The main evaluation does not establish that the guarantees are informative beyond the affine setting. Section 4.4 evaluates two ResNet-50 boundaries where the prediction head is affine. With affine decoders, the experiments confirm the exact additivity already implied by Proposition 1. The nonlinear decoder provides the main test of the broader attribution guarantee, but its bound exceeds the measured attribution error by two to three orders of magnitude. The fidelity bound is also approximately four to ten times the measured error. These findings characterize the conservativeness of the bounds, but do not demonstrate when the guarantees can meaningfully certify explanation accuracy or help choose between concept representations and attribution rules. Since the paper’s central value rests on its completeness guarantees, an informative application of those guarantees would materially strengthen the contribution.
[1] Fel, T., Boutin, V., Béthune, L., Cadène, R., Moayeri, M., Andéol, L., ... & Serre, T. (2023). A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems, 36, 54805-54818.
[2] Poché, A., Jacovi, A., Picard, A. M., Boutin, V., & Jourdan, F. (2025, July). Consim: Measuring concept-based explanations’ effectiveness with automated simulatability. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 5594-5615).
[3] Vielhaben, J., Bluecher, S., & Strodthoff, N. (2023). Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees. Transactions on Machine Learning Research (TMLR), 2023.
Questions for authors
-
Which concrete conclusion about existing explanation methods becomes available through the proposed framework beyond the encoder–decoder formulations and completeness results discussed in Section 5? A worked example where the new analysis changes how a method is assessed would help clarify the contribution.
-
Among the settings evaluated in Section 4.4, is there a case where the bounds provide an informative certificate or distinguish between methods in a practically meaningful way? In particular, how should the nonlinear attribution bound be used given its reported gap of two to three orders of magnitude?
-
How does the attribution analysis relate to methods that already satisfy completeness for nonlinear predictors? For example, Integrated Gradients, aggregated over concept blocks, can exactly recover the decoded predictor’s output difference under its standard assumptions (Sundararajan et al., ICML 2017). Clarifying why the proposed analysis focuses on the three selected rules would help establish its scope and added value.
[1] Sundararajan, M., Taly, A., & Yan, Q. (2017, July). Axiomatic attribution for deep networks. In International conference on machine learning (pp. 3319-3328). PMLR.
Engaging with the authors
Dear reviewer,
the authors have posted a reply to your review. Please have a look and interact with the authors as needed.
Thank you, Your AC