acceptodds
Under review as a conference paper at ICLR 2027

WSG-CBM: Weakly Supervised Grounding for Concept Bottleneck Models

Abstract

Concept Bottleneck Models (CBMs) improve interpretability by introducing human-understandable concepts as an intermediate representation between inputs and predictions. However, accurate concept prediction does not necessarily imply faithful concept reasoning. A model may correctly predict a concept while relying on visual regions that are semantically unrelated to that concept, causing a gap between concept correctness and concept grounding. We argue that concept grounding is a missing ingredient in concept bottleneck learning. To address this challenge, we formulate concept grounding as a weakly supervised concept-specific spatial evidence recovery problem. We propose the Weakly Supervised Grounded Concept Bottleneck Model (WSG-CBM), which learns concept-specific spatial evidence using only image-level concept supervision. To recover meaningful concept evidence under weak supervision, we further introduce grounding objectives based on evidence concentration, disentanglement, and reliance. Unlike existing grounded concept learning approaches, WSG-CBM requires neither concept localization annotations, segmentation masks, bounding boxes, nor external vision-language supervision. Experiments demonstrate that WSG-CBM consistently provides localized and faithful spatial evidence while maintaining competitive concept and classification performance. Our results show that concept grounding can emerge from concept supervision alone and highlight the importance of grounding as a first-class objective for faithful concept-based reasoning.

Primary area:
interpretability and explainable AI
Keywords:
Concept Bottlenecks, Explainability, Weak Supervision, Grounding, Faithfulness

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gYoMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper targets semantic grounding in Concept Bottleneck Models (does the model look at the right image regions for making the concept predictions). The paper tries to improve this grounding by introducing architectural changes and multiple loss terms (e.g. for disentanglement), and evaluates the approach on multiple datasets.

Critical strengths

Semantic grounding is an important problem in CBMs. The assumption CBMs implicitly make (that the concepts mean to the model what they should mean to the human) is vital and CBM works should not take it for granted.

Critical weaknesses

The paper’s central grounding claims are not sufficiently supported.

  • The approach uses a soft-attention mechanism for spatial pooling (Eq 3), similar to soft masking. Thus, every image region can still contribute to the concept prediction. There are no guarantees about which pixels contribute and which don't. To me, having such guarantees is very important for a paper centered on semantic grounding. Moreover, the authors discard the only loss that explicitly tries to enforce reliance on only the most important image regions (Eq 8) because it significantly hurts performance (Table 1). The final model therefore does not enforce reliance on the highlighted regions. There do appear to be existing CBM works where this reliance is strictly enforced (e.g. [1], which is also not mentioned in the related work). [1] also seems to have a similar story (regarding semantic grounding / alignment), which weakens part of the first contribution ("we identify concept grounding as a missing learning objective").
  • Concept-level IoU remains low even with the proposed regularization. Figure 3 supports this concern qualitatively: the spatial evidence maps for the displayed concepts overlap significantly, and the "leg color" map attends primarily to the bird’s upper body rather than its legs.
  • The statistical significance of the results is unclear. The number of seeds is not reported, and the tables don't provide standard deviations.

The paper is mathematically imprecise and insufficiently reproducible.

  • The text before Eq 5 states that entropy minimization is applied only to "present concepts", while the equation does not reflect this. Similar ambiguities occur in other equations.
  • The supplementary material misses important reproducibility details. It is unclear whether each baseline backbone is frozen or finetuned, how the CUB-pretrained ResNet was obtained, and what some of the hyperparameters are (e.g. batch size, number of epochs, learning rate schedules, ...).

[1] Colamonaco et al. Prototype-Grounded Concept Models for Verifiable Concept Alignment.

Questions for authors

I refer to the stated weaknesses.

5 comments

We thank the reviewer for the detailed feedback. We agree that the original manuscript did not sufficiently distinguish weakly supervised concept grounding from reliance-guaranteed grounding. Our goal is not to provide formal guarantees, but to investigate whether concept-specific spatial evidence can be learned directly from image-level concept supervision. We revised the manuscript accordingly, including additional comparisons, statistical analyses, qualitative evaluations, and reproducibility details.

Soft Attention and the Lack of Strict Reliance Guarantees. We thank the reviewer for raising this important point. We agree that methods providing formal reliance guarantees represent an important direction for grounded CBMs. However, obtaining such guarantees is substantially more challenging in the weakly supervised setting, where only image-level concept supervision is available, and no concept-level spatial annotations are provided.

Our work therefore studies a complementary problem: can concept-specific spatial evidence be discovered directly from image-level concept labels? To this end, WSG-CBM explicitly decomposes grounding into two stages. The concept-agnostic evidence branch identifies spatial regions likely to contain object-related evidence based on the learned representation, while the concept-specific evidence branch learns which regions support each individual concept. The resulting concept predictions are obtained by aggregating these learned evidence maps, allowing concept supervision to directly influence the localization process.

Accordingly, we do not claim that WSG-CBM provides formal guarantees regarding which pixels contribute to a prediction. Rather, we formulate concept grounding as a weakly supervised spatial evidence discovery problem and investigate whether meaningful concept-specific evidence can be recovered without localization supervision. Our objective is to find a practical balance between spatial alignment (whether the learned evidence corresponds to concept-relevant regions) and prediction relevance (whether the evidence contributes to the prediction), which we evaluate through both localization and faithfulness metrics. We revised the introduction to make it clear.

Exclusion of the Erase Loss (Eq. 8). We agree that Eq. (8) is the component most directly related to enforcing prediction reliance on highlighted regions. However, our experiments consistently show that incorporating Eq. (8) degrades not only concept and class prediction accuracy, but also localization and faithfulness metrics. In our setting, stronger reliance constraints derived solely from self-predicted evidence appear to reinforce incorrect localizations rather than improve concept grounding.

We therefore view Eq. (8) as an informative analysis component, but not a beneficial training objective for weakly supervised concept grounding.

More fundamentally, reliance on highlighted regions alone does not guarantee semantic correctness. Because erased regions are derived from the model's own predictions, they may themselves correspond to spurious evidence. Consequently, increased reliance can sometimes amplify incorrect localizations rather than improve semantic alignment.

Our objective is therefore not merely to increase sensitivity to highlighted regions, but to recover concept-specific spatial evidence that better aligns with concept semantics under weak supervision. We made this design rationale explicit in the revised manuscript.

PGCM and the Positioning of Our Contribution. We thank the reviewer for highlighting PGCM. We agree that PGCM provides an important notion of grounding through prototype-based verification. However, PGCM and WSG-CBM address different questions. PGCM assumes access to externally defined regions (segmentation masks or segmentation-derived parts) and explains concept predictions through prototype matching. In contrast, WSG-CBM investigates whether concept-specific spatial evidence can be discovered directly from image-level concept supervision without segmentation masks, predefined parts, or external localization priors. Notably, for CUB, PGCM treats the entire image as a single part and therefore does not explicitly recover concept-specific regions. Our objective is complementary: learning concept-specific spatial evidence under weak supervision.

PGCM focuses on verifiable concept alignment through prototype-based explanations and externally provided regions, whereas WSG-CBM focuses on recovering concept-specific spatial evidence directly from image-level supervision. In particular, PGCM does not explicitly recover concept-specific regions on CUB, as the entire image is treated as a single part. Consequently, the two approaches emphasize different aspects of concept grounding and should not be expected to be equivalent on localization metrics. We added corresponding discussion in related work in the revised version.

To further examine the relationship between reliance guarantees and concept grounding, we evaluated PGCM under the same experimental protocol. The full results are provided in Tables 7, 9, and 10 of the Appendix. Under the evaluated setting, WSG-CBM achieves stronger concept-level localization and faithfulness metrics than PGCM: concept-level localization metrics (IoU: 10.98 vs. 7.82, Dice: 17.48 vs. 12.87, C-IoU: 12.87 vs. 9.48) and faithfulness metrics (ROAD: 20.12 vs. 34.00, AG: 22.49 vs. 18.69), while maintaining competitive predictive performance (class accuracy: 74.46 vs. 72.72; concept accuracy: 88.38 vs. 91.22).

These results suggest that reliance guarantees and concept localization quality capture related but distinct aspects of concept grounding. In particular, stronger reliance guarantees and stronger concept localization should not be assumed to be equivalent objectives under weak supervision. Instead, the comparison indicates that meaningful concept-specific spatial evidence can be recovered without explicit region constraints, while simultaneously maintaining competitive recognition performance and prediction faithfulness.

Low Concept-Level IoU and Figure 3. We thank the reviewer for raising this concern. We agree that the absolute concept-level IoU values remain low and that the leg-color example in Figure 3 is imperfect.

However, recovering fine-grained concept regions from image-level concept supervision alone is inherently challenging, particularly on CUB, where many concepts correspond to small object parts. We therefore do not claim that WSG-CBM solves concept grounding or that every concept map perfectly matches the intended semantic region.

Instead, our claim is that meaningful concept-specific spatial evidence can be learned under weak supervision and can improve grounding quality relative to existing CBMs. Importantly, WSG-CBM consistently improves or remains competitive across localization, faithfulness, and predictive performance, rather than optimizing a single localization metric in isolation. We have revised the paper accordingly. Our contribution is not that concept grounding is solved, but that concept-specific spatial evidence can be learned from image-level concept supervision and can jointly improve localization quality, prediction faithfulness, and predictive performance under weak supervision.

We agree that Figure 3 alone is insufficient to support this claim. Accordingly, we added additional qualitative results (Figure 7-12 in the Appendix), including both successful and challenging examples, to provide a more representative assessment of the learned spatial evidence.

Statistical Significance. We thank the reviewer for raising this concern. All experiments were conducted using multiple random seeds. In the revised supplementary material, we now report the number of seeds together with mean ± standard deviation for the primary metrics (Tables 7, 9, and 10 in the Appendix). The resulting statistics confirm the same performance trends and do not alter the conclusions of the paper. Due to page limitations, we present the complete multi-seed results in the supplementary material while retaining the original tables in the main paper for readability. Should the reviewer prefer, we are happy to prioritize multi-seed statistics in the main manuscript and move less essential results to the appendix.

Mathematical Clarity and Reproducibility. We thank the reviewer for identifying these issues. The regularization in Eq. (5) is applied only to present concepts, as stated in the text immediately preceding the equation. The implementation and reported results are already consistent with this formulation. We agree that this condition could be made more explicit.

We also agree that the original supplementary material omitted several implementation details important for reproducibility. In the revised version, we provide the complete training protocol, including backbone initialization procedures, whether each backbone is frozen or fine-tuned, the source of the CUB-pretrained ResNet, learning rates and schedules, batch sizes, training epochs, optimizer settings, loss coefficients, and all hyperparameters required for reproduction.

Summary. Taken together, these revisions clarify the intended problem setting, improve the precision of the technical presentation, and address the reviewer's concerns regarding reproducibility and experimental transparency. Combined with the additional statistical analysis and clarification of our grounding claims, we believe the revised manuscript more clearly positions WSG-CBM as a framework for weakly supervised concept grounding rather than one providing formal reliance guarantees.

Reviewer_Yc5COpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper distinguishes concept correctness (whether a concept is present) from concept grounding (which part of the image provides evidence for that concept). It proposes WSG-CBM, which produces concept maps from spatial backbone features, obtains a spatial object evidence prior A shared by all concepts through SimPool, and aggregates them to obtain image-level concept predictions. The authors further explore losses intended to promote concentration, disentanglement, and reliance, and reported results on CUB, AWA2, and RIVAL10.

Critical strengths
  1. The problem is important and well motivated. Mapping concepts to certain regions of the input image makes the concepts visually inspectable and is valuable for assessing CBMs.

  2. The proposed architecture is straightforward. It also does not require additional spatial annotation beyond image-level concept labels, making it useful on existing datasets. Therefore, the proposed approach is potentially practical on existing data where concept labels already exist.

Critical weaknesses

1. Related work is not sufficiently discussed when framing the contribution of WSG-CBM.

The novelty of WSG-CBM is not well situated among related work. The compactness regularizer in Eq. (4) is also similar to the ones used by Xu et al. [1] and Zheng et al. [2]. The channelwise entropy loss in Eq. (6) has recently been used in SL-CBM [3]. The authors attributed neither of the two components to the original papers when introducing them.

References

[1] Xu, W., Xian, Y., Wang, J., Schiele, B. and Akata, Z., 2020. Attribute prototype network for zero-shot learning. Advances in Neural Information Processing Systems, 33, pp.21969-21980.

[2] Zheng, H., Fu, J., Mei, T. and Luo, J., 2017, October. Learning multi-attention convolutional neural network for fine-grained image recognition. In 2017 IEEE international conference on computer vision (ICCV) (pp. 5219-5227). IEEE.

[3] Zhang, H., Cheng, L., Wen, R., Zhang, Y., Zhang, L. and Hermanns, H., 2026, March. Sl-cbm: Enhancing concept bottleneck models with semantic locality for better interpretability. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 40, No. 44, pp. 38093-38101).

2. Incremental localization results, and major discrepancies with result reporting in general.

The localization results are not as strong as the narrative suggests. For example, in Table 4, WSG-CBM leads in CUB IoU by only 0.03 (11.11 vs 11.08), in CUB Dice by only 0.23 (17.65 vs 17.42), and trails in CUB C-IoU (13.12 vs 14.84). For CUB C-IoU specifically, WSG-CBM is NOT the best performing model, yet the authors bolded its results. This is misleading and should be fixed. Moreover, the visualization in Figure 3 only shows that WSG-CBM has more compact spatial evidence, yet it does not necessarily establish that the evidence is well-grounded in the image. "Leg" somehow corresponds to the bird's head, which raises more questions about the claimed improvements in localization.

Also, the main ECW-only row in Table 1 match a CPT+ECW configuration in Table 6, and the erase row in Table 1 match the weight 0.1 row in Table 6 when the stated default in the main text is 1. I'm really confused by this discrepancy, which significantly undermines the soundness of the ablation study and the paper in general. If a row presented as ECW-only actually includes CPT, it cannot isolate the contribution of ECW.

Questions for authors
  1. In general, the true novelty seems to be more about the aggregation method (see Weakness 1), which is not properly included in the ablation study. What is the incremental benefit of SimPool in comparison to average pooling and max pooling?

  2. Can you reconcile Tables 1 and 6? In particular, why does the main ECW-only row in Table 1 match a CPT+ECW configuration in Table 6, and why does the erase row in Table 1 match the weight 0.1 row in Table 6 when the stated default in the main text is 1? I'm really confused by this discrepancy, which also undermines the soundness of the ablation study and the paper in general.

  3. How does WSG-CBM compare with ProtoCBM?

2 comments

We thank the reviewer for the constructive comments and valuable suggestions. We address each concern as follows.

Related Work and Novelty. We appreciate the reviewer for pointing out the missing connections to prior work. We added citations to APN, MACNN, and SL-CBM to the corresponding losses.

More importantly, the primary contribution of this work is not the introduction of individual regularization losses. Rather, we formulate concept grounding as a weakly supervised localization problem and investigate whether concept-specific spatial evidence can be learned from image-level concept supervision alone. Existing CBMs primarily evaluate whether a concept is predicted correctly; our work asks the complementary question of whether the prediction is supported by spatial evidence consistent with the concept. To this end, WSG-CBM introduces a concept bottleneck design that explicitly associates concept predictions with learned spatial evidence maps through evidence-guided aggregation. The regularization terms are complementary components that facilitate this learning process rather than the central contribution.

Localization Results and Reporting Consistency. We agree that the original manuscript may overemphasize localization improvements on CUB. In the revised version, we moderate these claims and present WSG-CBM as a method that jointly improves grounding, faithfulness, and predictive performance rather than one that consistently achieves the best localization score under every metric. We also acknowledge that the leg-color example in Figure 3 is imperfect and represents a challenging failure case for weakly supervised concept localization. We include additional qualitative visualizations (Figures 7-12 of the Appendix) in the revised version to provide a more comprehensive assessment of grounding quality. We added a discussion on this in Section 5.

We also thank the reviewer for identifying the incorrectly highlighted C-IoU result. This was a presentation error. We carefully reviewed all tables and corrected the incorrectly highlighted results in the revised manuscript.

Our goal, however, is not to optimize a single localization metric in isolation, but to jointly improve spatial grounding, concept faithfulness, and predictive performance. While the concept-level IoU improvement over CEM on CUB is indeed small (11.11 vs. 11.08), WSG-CBM simultaneously achieves stronger class-level localization (IoU: 34.22 vs. 25.52; Dice: 48.18 vs. 36.50), improved concept and class prediction accuracy (88.61/74.63 vs. 87.86/72.07), and stronger faithfulness metrics compared with SL-CBM (concept-level AG: 22.51 vs. 11.10; concept-level ROAD: 20.67 vs. 29.96). These results suggest that the contribution of WSG-CBM extends beyond marginal gains in a single localization metric and instead provides a favorable balance across multiple dimensions relevant to interpretable concept learning.

Regarding the discrepancies between Tables 1 and 6, we thank the reviewer for carefully identifying this issue. After rechecking the experimental logs and configurations, we confirmed that the issue originates from an error during table compilation and result organization rather than from the underlying experiments themselves. The model checkpoints, training runs, and conclusions remain unchanged; however, several entries were inadvertently associated with incorrect configuration labels, including the ECW-only versus CPT+ECW configuration and the erase-loss coefficient. We apologize for this mistake, and we correct the affected entries with the precise configuration details, checking consistency across all tables in the revised manuscript.

Contribution of SimPool. We agree that the contribution of SimPool was insufficiently isolated in the original submission. Following the reviewer's suggestion, we conducted additional ablation experiments comparing SimPool against Average Pooling and Max Pooling (Table 8 in Appendix).

The results show that SimPool achieves the strongest localization performance, obtaining higher class-level IoU (34.22 vs. 30.94/30.81) and Dice (48.18 vs. 44.13/43.78) than both AvgPool and MaxPool. SimPool also achieves lower concept-level ROAD scores (20.67 vs. 24.40/23.52), lower class-level ROAD scores (40.63 vs. 82.15/77.30), and higher NEC-5 and ANEC, indicating stronger faithfulness and evidence quality. Although MaxPool achieves slightly higher classification accuracy, SimPool provides a substantially better trade-off between recognition performance, spatial grounding, and faithfulness. These results support our hypothesis that SimPool serves as an effective spatial evidence aggregation mechanism under weak concept supervision and contributes meaningfully to the overall performance of WSG-CBM.

The Comparison with ProtoCBM. Following the reviewer's suggestion, we additionally evaluated ProtoCBM to provide a more comprehensive comparison with prototype-based concept models (Tables 7, 9, and 10 in the Appendix).

The results show that WSG-CBM achieves higher concept and class prediction accuracy while maintaining competitive localization performance. For example, on CUB, WSG-CBM achieves 88.38 concept accuracy and 74.46 class accuracy, compared with 39.94 concept accuracy and 66.52 class accuracy for ProtoCBM, while preserving comparable concept localization quality. These results suggest that concept-specific spatial evidence learned through weak supervision can achieve a favorable balance between semantic grounding and predictive performance without requiring prototype-based reasoning.

Reviewer_RMPHOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper highlights the critical issue of misalignment between concept predictions and corresponding spatial evidence in existing CBMs and proposes a weakly supervised grounding framework for training CBMs with spatial consistency.

Critical strengths
  1. The authors identify a critical issue in CBM learning, where concept predictions may not align with their corresponding spatial evidence, and propose a novel method for learning spatially grounded CBMs.
  2. The authors have extensively evaluated the proposed method and grounding losses using metrics covering model accuracy, learned concept faithfulness, and localization of predicted concepts.
Critical weaknesses
  1. This work is missing results on larger datasets, including Imagenet and Places365, following [1,2]. These results are important to understand the scalability of the proposed approach as dataset size increases.

  2. In Figure 3, leg color is incorrectly grounded by WSG-CBM. Further, the paper lacks sufficient qualitative comparisons (apart from Figure 3) to visualize the spatial grounding of concepts, and the learned concepts used for class predictions.

[1] Yang, Yue, et al. "Language in a bottle: Language model guided concept bottlenecks for interpretable image classification." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.

[2] Oikarinen, Tuomas, et al. "Label-free concept bottleneck models." arXiv preprint arXiv:2304.06129 (2023).

Questions for authors

My primary concerns are the lack of results on larger datasets and lack of qualitative results. Please see weaknesses for more details.

1 comment

We thank the reviewer for the constructive feedback and for recognizing the importance of concept grounding in CBMs.

Regarding scalability, we agree that larger-scale evaluations would further strengthen the paper. However, WSG-CBM does not introduce computationally intensive operations beyond standard CBM architectures. It leverages existing visual backbones and adds lightweight grounding modules, resulting in computational complexity comparable to prior CBMs. We further note that, in concept bottleneck learning, practical limitations often arise from the availability of concept annotations rather than computational scalability itself. Our evaluation on RIVAL10, a benchmark derived from ImageNet, provides initial evidence that the approach generalizes beyond small fine-grained datasets. We highlight the connection between RIVAL10 and ImageNet in the experimental setting and add a dedicated discussion in Section 5 of the revised manuscript on the scalability and annotation requirements of WSG-CBM.

Regarding qualitative evaluation, we agree that the original manuscript did not provide sufficient visual evidence for assessing concept grounding. In response, we have added additional qualitative visualizations in Figure 9 of the Appendix (see supplementary materials), showing representative examples across multiple images and concepts (breast, head, leg, tail, and wing), together with the corresponding object-level saliency maps. These examples provide a more comprehensive assessment of the learned concept-specific evidence and illustrate both the strengths and limitations of weakly supervised concept grounding.

Regarding the leg-color example in Figure 3, we agree that this example remains imperfect and highlights a limitation of current weakly supervised concept grounding methods. Our intention was not to claim perfect concept localization, but rather to illustrate the type of concept-specific spatial evidence learned by WSG-CBM under image-level supervision only. We agree that the leg-color example highlights a challenging failure case. To provide a more representative assessment of grounding quality, we now include additional qualitative examples (Figure 7-12 of the Appendix in the supplementary material) covering both successful and challenging cases.

Comments on the paper

3 comments

We thank the reviewer for the careful follow-up and for raising the score. We agree that our revised wording may still overstate the evidence.

As the reviewer correctly points out, the model without grounding objectives achieves the highest concept and class accuracy in Table 1. Therefore, our results do not support the claim that the proposed grounding objectives universally improve predictive performance. Rather, they reveal a trade-off between optimizing purely for predictive accuracy and encouraging concept predictions to rely on spatially meaningful evidence.

The role of the grounding objectives is to constrain the solution space by encouraging concepts to be supported by compact, distinct, and prediction-relevant spatial evidence. Such constraints can slightly reduce concept and class accuracy while improving grounding and faithfulness metrics. From this perspective, the reduction in concept and classification accuracy is not a contradiction to our findings, but reflects the cost of imposing additional structure on the learned concept representation.

We also agree that the current discussion may place too much emphasis on NEC-style metrics. Our intention was not to dismiss concept or class accuracy, but rather to highlight that correctness alone does not guarantee faithful concept reasoning. We therefore view concept accuracy, class accuracy, grounding metrics, and faithfulness metrics as complementary dimensions that capture different aspects of concept quality.

In response to the reviewer's suggestion, we have revised the manuscript to explicitly discuss this trade-off and moderate the corresponding claims. Instead of claiming that WSG-CBM jointly improves grounding, faithfulness, and predictive performance, we now describe it as achieving a more favorable balance between grounding quality, faithfulness, and predictive performance under weak supervision.

We thank the reviewer for highlighting this distinction, as it led to a more accurate presentation of the contribution.

Thanks for your response. My remaining concern is addressed. I'll maintain my positive rating.

AuthorsOpenReview ↗

Final Author Summary

We thank all reviewers for the detailed feedback and constructive discussion. The rebuttal process substantially improved both the empirical evaluation and the positioning of the paper.

Following reviewer suggestions, we:

  • corrected the reporting inconsistencies and clarified all ablation configurations;
  • expanded the related-work discussion (APN, MACNN, SL-CBM, PGCM, ProtoCBM);
  • added SimPool ablations against AvgPool and MaxPool;
  • added ProtoCBM and PGCM comparisons, including both whole-image and part-prototype variants;
  • reported multi-seed results (mean ± std), expanded reproducibility details, and clarified implementation choices;
  • added extensive qualitative analyses (Figures 7–13), including both successful and failure cases;
  • added dedicated discussions on scalability, limitations, and future directions.

The reviewers broadly agree on the importance of concept grounding and the relevance of the problem. Following the rebuttal, concerns regarding reporting consistency, missing ablations, statistical robustness, reproducibility, and qualitative analysis were addressed through additional experiments and manuscript revisions. Notably, Reviewer Yc5C indicated that the major concerns were addressed and raised the score accordingly.

The remaining discussion primarily concerns the strength of grounding guarantees achievable under image-level supervision alone. In response, we have explicitly narrowed our claims and now position WSG-CBM as a framework for weakly supervised concept grounding, rather than one providing formal reliance guarantees. We fully agree that stronger mechanisms for enforcing and verifying concept reliance remain an important open research direction.

At the same time, the central question of the paper remains unchanged: Can concept-specific spatial evidence be learned directly from image-level concept supervision, without segmentation masks, localization annotations, predefined parts, prototype supervision, or vision-language priors?

Across CUB, AWA2, and RIVAL10, our results provide evidence that meaningful concept-specific spatial evidence can be learned under this weak-supervision setting while achieving a favorable balance between localization quality, faithfulness, intervention utility, and predictive performance.

We therefore believe the paper contributes a well-motivated and empirically supported step toward weakly supervised concept grounding in CBMs under realistic annotation constraints. We thank the reviewers again for helping us strengthen the manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.