Intervening in Concept Bottleneck Model via Causal Effects from Pixel to Concept

Concept bottleneck models (CBMs) employ human-interpretable concepts for predictions, enabling corrective interventions in neural networks. However, current intervention methods are usually limited to the concept bottlenecks encoded by features or their corresponding weight matrices, leading to inefficient interventions, limited interpretability, and dependence on human priors that restrict generalization. To address this, we propose CBM-PCCEI, a Pixel-to-Concept Causal Effect Intervention method for CBMs. It utilizes causal effects hierarchically to achieve multi-level intervention, improves effectiveness, and provides a more easily understood explanation of reasoning and intervention. To generalize improvements beyond the intervened cases and reduce dependence on human priors, we integrate the interventions with knowledge distillation, transferring the improved behavior to the base model. Evaluations on Flower-102, Aircraft, CUB-200, CIFAR-10, and CIFAR-100 with NFResNet-50 and DaViT show that our method improves validation accuracy by up to 11.01% with an average gain of 6.62% via single patch intervention, and achieves up to 3.82% with an average of 1.48% testing accuracy gain under generalization settings, outperforming both vanilla fine-tuning and state-of-the-art intervention strategies.