- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies spatial grounding in CBMs, arguing that concept prediction accuracy alone is insufficient to establish whether a model relies on the intended visual evidence. The authors propose an evaluation framework based on spatial sufficiency and necessity, together with a grounding score and a Spatial Concept Influence Matrix. Through experiments on CUB and RIVAL 10, they demonstrate that models with similar concept accuracy may show different spatial grounding properties.
Critical strengths
-
The paper studies an important and relevant question concerning the faithfulness and interpretability of CBMs.
-
The distinction between concept accuracy and spatial grounding is meaningful to the community and provides an interesting perspective on evaluating concept representations.
Critical weaknesses
1. Limited novelty and contribution
While the motivation is interesting, I find the methodological contribution to be relatively incremental. The proposed framework mainly builds on existing ideas of sufficiency, necessity, and perturbation based evaluation. In particular, the grounding score is defined as
which is a rather simple combination of the two proposed measures.
The main finding, that accurate concept predictions do not necessarily imply reliance on the intended spatial evidence, is valuable but also somewhat expected given existing work on concept faithfulness and localization. Similar concerns have already been raised in the literature [7, 8], including empirical evidence that CBMs may predict concepts correctly while relying on irrelevant or spuriously correlated features.
I am therefore not fully convinced that the paper provides enough new insights beyond an additional empirical characterization of known limitations. I think the authors should explain more clearly what new understanding their framework provides compared to previous studies of concept locality and faithfulness.
2. Limited experimental scope and generalizability
Although several architectures and backbones are evaluated, the experiments are restricted to two visual attribute datasets, CUB and RIVAL 10. I find this somewhat limited, especially since the authors present spatial grounding as a complementary dimension for evaluating the interpretability and faithfulness of visual CBMs more generally (Sections 1 and 3).
Some of the analyses are also quite narrow. For example, the comparison of training strategies is conducted with only one CBM and backbone configuration. It is not clear whether the conclusion that training strategy has little influence on grounding would hold across other architectures.
I would have liked to see experiments on additional datasets and possibly different settings, to better understand how general the findings are. Evaluating several existing architectures is useful, but I do not think this alone provides sufficient evidence for the general conclusions of the paper.
Another concern is the dependence on the selected spatial regions. Since the metrics are computed over a region , their values may depend on the spatial coverage
The authors acknowledge this issue, but I think a more systematic evaluation of how region size and mask quality affect the results would strengthen the experimental analysis.
3. Insufficient validation of the proposed metrics
I think the proposed metrics require further validation. In particular, the authors perform interventions on the intermediate feature representation by applying a spatial mask , such that
While this avoids directly masking the input image, setting features outside to zero may still produce representations that are outside the distribution seen during training. I am not convinced that performing these interventions in feature space necessarily avoids the distribution shift problem, as suggested by the authors.
I also have some concerns about the mathematical formulation of the proposed measures. For example, spatial sufficiency is defined as
\frac{|f_i(F_R)-f_i(F_{\emptyset})|} {|f_i(F)-f_i(F_{\emptyset})|+\epsilon},$$ with the result clipped to $[0,1]$. Due to the absolute value in the numerator, a region may receive a high sufficiency score even when the prediction changes in the opposite direction relative to the fully masked baseline. For example, if $$f_i(F)=0.9,\quad f_i(F_{\emptyset})=0.5,\quad f_i(F_R)=0.1,$$ the sufficiency score is approximately $1$, although the prediction changes from $0.9$ to $0.1$. This seems inconsistent with the intended interpretation of sufficiency as preserving the original concept prediction. A similar concern applies to the interpretation of the necessity score. I think these issues deserve more discussion and ideally some controlled experiments. For example, comparing different masking baselines, testing the sensitivity to region size, or validating the metrics on synthetic settings where the relevant spatial evidence is known. Overall, I would like to see stronger evidence that the proposed scores measure the intended notion of spatial grounding rather than being affected by the particular intervention procedure. ### 4. Missing relevant related work I also think the related work section could be improved. There are several relevant papers studying spatially grounded or locality aware CBMs that are not discussed. For example, SALF CBM [1], SEG MIL CBM [2], Locality aware CBM [3], DCBM [4], and CHILI [5] all investigate different aspects of spatial concept representations, concept localization, or the relationship between concepts and visual evidence. I understand that these works are not necessarily trying to solve the exact same problem, but I think they are relevant to the motivation of this paper. It would be helpful to better explain how the proposed evaluation framework relates to these approaches, and what limitations of existing methods it addresses. Some of these methods could also be interesting additions to the experimental comparison. In addition, the proposed sufficiency and necessity measures should be better connected to existing literature on faithfulness evaluation, such as ERASER [6], which already studies related ideas through sufficiency and comprehensiveness. While the setting is different, the underlying idea of measuring how a prediction changes when evidence is retained or removed is closely related. I think a more complete discussion of these works would help clarify what is actually novel in the proposed formulation, and better position the paper within the existing literature. ### 5. Limited practical implications The paper introduces an evaluation framework, but I find that the practical implications remain somewhat unclear. Beyond showing that different models achieve different grounding scores, it is not clear how these measurements can be used to improve CBMs or inform their design. For example, if a model achieves high concept accuracy but low spatial grounding, what should we do differently when training or selecting that model? It would be interesting to see whether optimizing for higher grounding actually improves concept faithfulness, or whether the proposed scores correlate with improved robustness to spurious features. I understand that the goal of the paper is primarily evaluation rather than proposing a new training method. Still, I think showing at least one practical use case for the framework would make the contribution more convincing. ### 6. Reproducibility and code availability As far as I could tell, the authors have not released the code for the proposed framework or experiments. Given that the main contribution is an evaluation methodology, I think making the implementation available is particularly important for reproducibility. ## References [1] [Show and Tell: Visually Explainable Deep Neural Nets via Spatially Aware Concept Bottleneck Models](https://openaccess.thecvf.com/content/CVPR2025/html/Benou_Show_and_Tell_Visually_Explainable_Deep_Neural_Nets_via_Spatially-Aware_CVPR_2025_paper.html). Itay Benou and Tammy Riklin Raviv. CVPR, 2025. [2] [From Segments to Concepts: Interpretable Image Classification via Concept Guided Segmentation](https://arxiv.org/abs/2510.04180). Ran Eisenberg et al. arXiv, 2025. [3] [Locality aware Concept Bottleneck Model](https://arxiv.org/abs/2508.14562). Sujin Jeon et al. arXiv, 2025. [4] [DCBM: Data Efficient Visual Concept Bottleneck Models](https://proceedings.mlr.press/v267/prasse25a.html). Katharina Prasse et al. ICML, 2025. [5] [Enhancing Concept Localization in CLIP based Concept Bottleneck Models](https://arxiv.org/abs/2510.07115). Rémi Kazmierczak et al. TMLR, 2026. [6] [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://aclanthology.org/2020.acl-main.408/). Jay DeYoung et al. ACL, 2020. [7] [Do Concept Bottleneck Models Learn as Intended?](https://arxiv.org/abs/2105.04289). Andrei Margeloiu et al. arXiv, 2021. [8] [Do Concept Bottleneck Models Respect Localities?](https://arxiv.org/abs/2401.01259). Naveen Raman et al. arXiv, 2024.Questions for authors
-
Regarding Weakness 3, could the authors clarify why zero masking in the intermediate representation is expected to avoid distribution shift? Have alternative masking strategies been considered?
-
Regarding Weakness 3, how do the authors justify the use of absolute differences in the sufficiency and necessity definitions, particularly when the masked prediction changes in the opposite direction?
-
Regarding Weakness 2, is there additional evidence supporting the generality of the conclusions across datasets and training configurations beyond those currently evaluated?
-
Regarding Weakness 4, could the authors clarify how the proposed framework differs from existing spatially grounded CBM approaches, particularly SALF CBM and SEG MIL CBM, and what additional insights it provides?
4 comments
Relationship to SALF-CBM and SEG-MIL-CBM. We agree that these approaches are relevant and will expand the related work discussion. SALF-CBM and SEG-MIL-CBM investigate spatially informed concept learning and localization through their respective architectural and learning strategies. Our work addresses a complementary question: given a trained CBM, does the spatial region associated with a concept actually support the prediction made by its concept predictor?
Our framework is an evaluation methodology rather than another spatially constrained architecture. It can therefore be applied across different CBM designs to examine whether their learned representations exhibit sufficient, necessary, redundant, or distributed spatial evidence. We also discussed DCBM, Locality-aware CBM, CHILI, and ERASER to better position the work in relation to spatial concept learning and faithfulness evaluation in the related work section of the revised version.
Practical implications. Our framework is intended as a diagnostic tool, not as a claim that maximizing the grounding score necessarily improves robustness or faithfulness. A high concept accuracy combined with low grounding indicates that correct predictions do not necessarily rely on the intended spatial evidence. The separate sufficiency and necessity scores further help identify whether this results from insufficient regional evidence or redundant/distributed evidence. Such diagnoses can inform model selection and motivate different architectural interventions. We will clarify these practical implications while acknowledging that establishing whether grounding optimization improves robustness requires further experiments.
Reproducibility. We agree that code availability is important for an evaluation framework. To preserve anonymity during double-blind review, we cannot publicly release the repository at this stage. We commit to releasing the implementation and experimental code upon acceptance and providing the repository link in the final version.
Thank you for the detailed response and the revised version.
What has been addressed. The related work is much improved, especially the new discussion of spatially-aware CBMs and ERASER. The clarifications on clipping and standard deviations, and the more careful wording of the main findings, are also helpful. The explanation given to Reviewer eVPK about where the interventions are applied would be worth adding to the main text.
What remains.
-
Validation of the metric. Table 3 does not convince me that the scores measure grounding rather than the intervention itself. Random regions reach almost the same sufficiency as the annotated part masks, and several other regions (object masks, saliency maps) give higher grounding than the part masks. A controlled setting where the evidence location is known, and an alternative masking baseline, would help.
-
The formulation. Equations 2 and 3 still use the absolute value , so the direction of the change is ignored. Appendix C.1 notes that for SL-CBM, nonzero necessity occurs only when , so effectively reduces to necessity. This suggests the scores may depend strongly on how the concept head aggregates spatial features, which deserves discussion. In addition, according to the Figure 1 caption, the distributed regime (high necessity, low sufficiency) used to motivate the joint formulation is never observed.
-
Distribution shift. The response acknowledges that feature masking does not remove distribution shift. However, the paper still states that it avoids it (l. 090-091) and now criticizes input perturbations for the same issue.
-
Scope, practical use and code. No new experiments were added. The training-scheme comparison covers only the standard CBM (Table 9), and I could not find the mask-quality limitation mentioned in the response. No practical use case is shown, and an anonymized code repository would be compatible with double-blind review.
Overall, the revision improves the presentation and makes the claims more careful, and I thank the authors for engaging openly. However, my main technical concerns remain, so I will keep my score.
Thank you for the careful reading and for the detailed follow-up. We appreciate the recognition that the revised manuscript clarifies the relationship to prior work and improves the presentation of the framework.
Validation of the metric. We agree that the proposed scores are defined through interventions and therefore quantify the effect of retaining or removing spatial evidence on concept prediction. However, this is intentional rather than incidental. We define a concept as spatially grounded when the corresponding concept prediction is supported by the spatial evidence that is semantically associated with that concept. Under this definition, sufficiency and necessity provide an operational test of whether the concept predictor relies on the expected spatial evidence.
In our primary evaluation, ground-truth concept masks provide a controlled reference for where concept evidence is expected to reside. Spatial grounding therefore measures whether the concept predictor utilizes the corresponding spatial feature information in a manner aligned with human annotations. We agree that stronger validation through synthetic settings with known causal evidence or alternative intervention baselines would further strengthen the study.
However, we respectfully disagree that the current results suggest the metrics are dominated by the intervention procedure itself. For example, in Table 3(B), annotated part masks achieve substantially higher sufficiency than random regions (0.921 vs. 0.867), despite comparable spatial coverage. The proposed scores consistently rank semantically meaningful regions above random regions and distinguish architectures designed to encourage spatial alignment from standard CBMs, providing empirical evidence that they capture meaningful properties of spatial support rather than arbitrary intervention artifacts.
Our framework intentionally reveals cases where alternative regions (e.g., object masks or attribution-derived regions) achieve higher grounding than the annotated concept region. We do not interpret this as a failure of the metric; rather, it indicates that the model relies on spatial evidence that differs from the evidence designated by the human annotation. This is precisely the type of model–human misalignment that the proposed grounding analysis is intended to expose.
The formulation. Indeed, our intent was to measure the magnitude of spatial dependence rather than directional agreement with the original prediction. We agree that this choice warrants further discussion, particularly for architectures such as SL-CBM and in settings where necessity dominates the combined grounding score. It is indeed related to the design of concept predictor, especially SL-CBM derives concept predictions through cross-attention between spatial features and concept embeddings from a concept bank. As a result, concept evidence can be aggregated from multiple spatial locations, which naturally leads to high sufficiency but comparatively low necessity. We view this behavior as an informative property of the model rather than a failure of the metric, as it reveals how concept evidence is distributed within the representation. We have therefore revised the manuscript to clarify the interpretation and limitations of the current formulation and note that direction-aware variants are an interesting direction for future work.
While the distributed regime (high necessity, low sufficiency) is not observed in our current experiments, the redundant regime (high sufficiency, low necessity) is consistently observed across multiple architectures and datasets. This observation itself motivates the joint formulation, as sufficiency and necessity capture complementary aspects of concept support and reveal forms of evidence redundancy that would not be visible from either measure alone.
Distribution shift. Our claim is not that feature-space interventions eliminate distribution shift, but rather that they avoid some of the low-level perturbations introduced by image-space masking and propagated through the entire backbone, while allowing interventions to be performed directly on the representation consumed by the concept predictor. We have revised the manuscript accordingly to clarify that feature-space masking should be viewed as a pragmatic intervention strategy rather than a guarantee of in-distribution evaluation.
Scope and reproducibility. We agree that broader empirical validation would strengthen the conclusions and have revised the discussion to better qualify the scope of our findings. We will also release the implementation upon acceptance.
We sincerely appreciate the detailed feedback. The discussion has helped us substantially improve both the presentation of the framework and the clarity of its assumptions and limitations.
We thank the reviewer for the detailed feedback. We appreciate the recognition of the importance of the problem and agree that the relationship to prior work, the interpretation of the proposed metrics, and their practical value should be clarified. We address the concerns below.
Novelty and distinction from prior work. We agree that our framework builds on established ideas of sufficiency, necessity, and perturbation-based evaluation. Our contribution is not to introduce these principles independently, but to adapt and combine them to characterize the spatial support of concept predictions within the representation consumed by the concept predictor.
The key distinction from concept locality is the joint evaluation of spatial sufficiency and necessity. Locality or localization identifies regions associated with a concept, but does not necessarily establish whether the region alone retains the concept evidence or whether the prediction depends on that region. These properties are not equivalent. For example, a region can be sufficient but not necessary when similar evidence is redundantly encoded elsewhere. Conversely, a region can be necessary but insufficient when evidence is distributed across multiple regions. The resulting support regimes provide information that cannot be obtained from localization or concept accuracy alone. We strengthen the discussion of Margeloiu et al. and Raman et al. and clarify that our framework complements prior concept-faithfulness and locality analyses by explicitly characterizing these spatial support regimes.
More broadly, our framework reveals that concept support is multi-dimensional. Models with similar concept accuracy or locality can exhibit fundamentally different support regimes, such as high sufficiency/low necessity or low sufficiency/high necessity, which are not visible under existing evaluations.
Experimental scope and generalizability. We acknowledge that the evaluation is limited to CUB and RIVAL-10 and that broader validation across datasets would strengthen the conclusions. Nevertheless, our experiments cover multiple bottleneck architectures, backbone families, and training strategies. In particular, sequential and joint training produce similar grounding scores on both CUB and RIVAL-10 (Appendix Table 9). We have therefore revised the manuscript to present these observations as evidence within the evaluated settings, rather than as general conclusions that necessarily extend to all CBM configurations.
Regarding spatial coverage, our evaluation includes regions derived from ground-truth masks, attribution maps, random regions, and object masks. These comparisons use controlled spatial coverage, and region size is reported in Table 3(B). We agree, however, that this does not constitute a systematic study of mask quality, such as sensitivity to annotation noise, mask erosion, or dilation. We clarified this limitation and qualify the conclusions accordingly.
We would be grateful if the reviewer could indicate specific datasets, CBM variants, or evaluation settings they believe are most critical and currently missing, as this would help us prioritize the most informative extensions in the revision and future work.
Validation of the proposed metrics.
Despite these limitations, the proposed metrics exhibit consistent behavior across controlled evaluations. They assign substantially higher grounding to ground-truth concept regions than to random regions and reliably distinguish architectures designed to promote spatially aligned concepts from standard CBMs (Table 3). Although this cannot provide a formal guarantee that the metrics perfectly capture grounding, it offers empirical evidence that they measure meaningful properties of spatial concept support rather than artifacts introduced by the intervention mechanism.