Beyond Concept Accuracy: Spatial Grounding in Concept Bottleneck Models

Concept Bottleneck Models (CBMs) aim to make predictions interpretable by decomposing reasoning into human-interpretable concepts and downstream decisions. While existing CBM research has extensively studied whether task predictions faithfully follow concept representations, it has paid less attention to how concept predictions depend on spatially localized information. Consequently, concept accuracy alone provides limited insight into the spatial evidence supporting a concept. To this end, we introduce a spatial grounding framework for concept representations that characterizes concept support through two complementary properties: spatial sufficiency and spatial necessity. Spatial sufficiency measures whether a region contains enough information to preserve a concept prediction, while spatial necessity measures whether removing that region disrupts the prediction. Together, these measures characterize how concept predictions depend on spatially localized information within the concept-generating representation. Across CUB-200-2011 and RIVAL-10, we find that concept and task accuracy does not determine spatial grounding; comparable predictive performance yields substantially different grounding behavior. Grounding depends on both the concept bottleneck and backbone representation, while joint concept-task optimization do not substantially change grounding. Finally, we show that attribution-based localization and spatial grounding capture distinct properties of concept representations. These findings establish spatial grounding as a complementary dimension for evaluating the interpretability and faithfulness of visual CBMs.