acceptodds
Under review as a conference paper at ICLR 2027

Hoeffding Concept Bottleneck Models with Applications to Overhead Images

Abstract

Explainability of deep learning algorithms is critical for computer-vision applications with high-stakes decisions. Concept bottleneck models (CBM) have recently shown promising performance to provide explainable and accurate predictions for classification problems, based on a bottleneck of high-level concepts. Existing CBM methods rely on a linear aggregation of the concept scores to compute predictions. However, a large number of concepts is often used in this linear approach, which undermines explainability and favors information leakage. In general, the underlying relation between concepts and output logits is not linear. Therefore, we introduce Hoeffding Concept Bottleneck Models (HCBM), which build on the Hoeffding functional decomposition of gradient-boosted trees to provide non-linear and sparse aggregations of concept scores, and generate compact predictions using prime implicants. HCBM have strong theoretical properties of robustness to interconcept leakage, and outperform sparse linear CBM in practice, as shown in extensive experiments. Beyond classification, HCBM can be adapted to object detection, and we focus on a challenging case with overhead images to show the high performance of HCBM in these settings.

Primary area:
interpretability and explainable AI
Keywords:
Concept Bottleneck Models, Hoeffding functional decomposition, information leakage, additive models, computer vision, object detection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Pg1jOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Proposes HCBM, swaps linear aggregation in CBMs for sparse nonlinear one.CLIP/SigLIP concept scores -> per class XGBoost -> TreeHFD, select concepts by importance,monotone constraints, retrain. prime implicants for per-prediction explanations, plus a result that irrelevant concepts get zero components (under full Hoeffding decomp). Exps on natural+remote sensing, injected leakage, interventions in appendix, xView w/ RTMDet logits (detector faithfulness, not localizaton)

Critical strengths

Big gains at small budgets, and concepts/features fixed so its really the aggregation. CLIP RESISC NEC=3 0.673->0.816, ImageNet NEC=5 0.412->0.605

not just qualitative plots — MetaShift deletion HCBM 0.848->0.831 vs linear 0.850->0.781. nice, tho doesn't show concepts are semantically right/causal

Critical weaknesses

prime implicant code can return non-implicants. hcbm.py L414-423 sums h_values over ALL concepts (minus deleted ones) when checking removal,so concepts never in the candidate set count as fixed.

counterexample: u_j=1[z_j>=0.5], Δ=-4+3u1+2.1u2+2u3. z=(1,1,1) gives 3.1, candidates {1,2}, code returns {1}. but Δ(1,0,0)=-1 → flips. reproduced it. so sufficiency fails not just minimality. linear CBM code looks the same so both length stats are suspect

also L348-369 uses 1000pt grid for the min, can miss narrow gaps btw splits. got a 2nd invalid case from that too

Theory doesnt really transfer. Thm 2.2/F.1 need full decomp (all orders, density bounds), exps use main effects only — truncated interactions can leak into correlated proxies. E.2 admits RESISC/ImageNet break the density assumption, fair, but then cant use the thms there.

eq(2) is relative variance per class but code (L257-258,L150-168) pools raw variance acros classes?? which one was used

Questions for authors

Tables 10-11 / examples made with the attached compute_prime_implicants? pls fix, use exact mins at breakpoints, report sufficiency checks+new lengths for both models. full |S(X)| per prediction would help

w/ interaction_order=1 what of F.1 still holds,extra additivity assumption? narrowing claims is fine. also eq(2) vs code

xView CV is random KFold on crops, no grouping by image/region. same for Table 2? overlapping crops across folds → want grouped results

3 comments

Thank you for the thorough verification of our code, and the identified bugs and typos. We corrected all of them and reran experiments to show that they all have no impact on performance metrics, except the post-treatment of prime implicants which slightly underestimated their size, but leaves the claims about prime implicants efficiency untouched. We also added a new theoretical result and additional experiments for xView. We hope that these improvements address your concerns, and that you will revise your score. (Changes are in blue font in the attached revised article.)

W1. Code verification for prime implicants. There is indeed a bug in the code for the post-treatment of prime implicants. We fixed the code and rerun experiments of Tables 10 and 11 (now 11 and 12 in the revised article), to show that the sizes of prime implicants are slightly higher for both linear CBM and HCBM, but are still similar to the original values. Therefore, the claim that prime implicants efficiently simplify HCBM explanations still hold. Thank you for pointing this out.

Additionally, we also added the exact computation of minimum of the difference of TreeHFD main effects. Since TreeHFD main effects are piecewise constant, we simply need to retrieve all cut points of all trees for each concept, build an adaptive grid with one point between two consecutive cut points, and take the minimum over this set of points for each concept. The results of this improvement combined with the bug fix are displayed in the updated Tables 11 and 12. The impact of computing exact minimum compared to the minimum approximation with the regular grid is probably negligible for the prime implicant sizes, as opposed to the bug fix.

W2. Code mismatch for concept importance. Indeed, there is a typo in Equation (2), where the right-hand side should be only α\alpha, as implemented in the code and used in all the theoretical analysis of Appendix F2. Thank you for pointing this out. We thus corrected Equation (2) in the revised article. Also notice that this normalization has almost no effect in practice, since all logits have close variance for balanced classification. We tested to add this normalization in the code, and then reproduced the accuracy performance of Table 4. However, this normalization should not be included, because if a given class has a very small number of relevant concepts, noisy ones may be selected in specific cases.

W3. Theory assumptions. We provide the additional Theorem F3 in Appendix F2 to show that the theoretical guarantees of Theorem F1 still holds when TreeHFD is fit with only main effects. See W1 in the rebuttal of Reviewer Lom8 for the discussion about this new result. See also the last question in the rebuttal of Reviewer d21A above for the discussion about the positive density assumption.

Q1. Implementation of prime implicants. See W1.

Q2. Theory assumptions. See W3.

Q3. Object detection. See W2 about object detection in the rebuttal of Reviewer Lom8.

Thank you for the detailed response and revisions. The correction to Equation (2) resolves my concern about the mismatch between the stated importance measure and the implementation. I also appreciate the scene-grouped xView experiments in Tables 20–21, which address my concern about crops from the same source image appearing across folds. The additional Theorem F.3 is a meaningful extension to the main-effect setting.

My remaining concerns are primarily about verifying the corrected prime-implicant results and clarifying the scope of the guarantees.

1. Verification of the corrected prime-implicant implementation.
The cutpoint-based minimum computation described in the revised Appendix C is an appropriate approach. However, my original concern was that the returned sets could fail to be sufficient, rather than only having underestimated sizes. Updated average sizes alone do not establish that this issue is resolved.

Could you provide the corrected implementation or a minimal patch for both HCBM and the affected linear-baseline routine? The code archive available in the materials I checked appears to contain the original implementation, so this may simply be a version mismatch.

Please also provide regression checks for the counterexample in my review and a narrow-interval case missed by the uniform grid. For a representative evaluation set, please report the number of returned explanations checked and any sufficiency violations, using exact worst-case margins against every competing class while allowing all concepts outside the returned set to vary. This would directly verify the property that was affected by the bug.

2. Full explanation size per prediction.
Tables 11–12 continue to report the average number of implicant concepts per logit. This differs from the total number of distinct concepts in the sufficient explanation, ∣S(X)∣|S(X)|, which I requested in the original review. Could you additionally report the full-set size per prediction, with a mean and distribution summaries? This would help assess the compactness of the complete explanation, particularly in the multiclass settings.

3. Scope of the main-effect guarantee.
I appreciate Theorem F.3 and consider it substantive progress. Its additional assumption concerns uniformly small interaction components in each individual tree, which is stronger than observing little change in predictive performance when interactions are omitted. I also share the concern raised in the discussion about distinguishing variance ordering averaged over the training data from the empirical ranking of one fitted model. Representative quantitative evidence would be useful; alternatively, the corresponding claims should clearly state their conditional scope and distinguish them from a guarantee for the implemented empirical ranking.

For now, I am maintaining my rating of 2 (Weak rejection), pending clarification of these remaining points. I recognize the empirical improvements at small concept budgets and the substantial work done in the revision, and I remain open to increasing my rating once the corrected explanation procedure is verifiable and the claims are appropriately scoped.

Thank you for your quick reply to our rebuttal, and for highlighting that we addressed many of your initial concerns. We address below the remaining points.

1. Verification of the corrected prime-implicant implementation. Our new implementation of prime implicants now provides a list of concepts for each HCBM prediction which satisfies the property of sufficiency, and no strict subset of the list also satisfies it, because we compute exact minimum of main effect differences. We checked that our implementation now outputs valid prime implicants for the counterexample of your initial review and for cases with narrow intervals.

We indeed did not provide the updated code in the first revision, but we have now updated the attached Supplementary material with the new code version.

We specify two details: when the two top logits are tied for a given prediction (very rare case), no implicant exists by definition, and we thus consider that no concept can be dropped by convention, and all model concepts are returned. We also corrected a typo in the implementation of prime implicants for linear CBM in the specific case of two classes (with only one logit, as in Waterbirds dataset), and updated the metrics.

2. Full explanation size per prediction. In Table 13 of Appendix G3, we now report the average size of prime implicants for each prediction, along with the standard deviation of the size distribution across all predictions. Thank you for this suggestion, since this metric seems easier to read than our initial one. The results confirm the high efficiency of prime implicants to strongly reduce the number of required concepts to explain a given prediction. For example, for CIFAR-10 with NEC at 10, 100 concepts are involved in the CBM, and prime implicants have an average size of 31 for HCBM and 33 for linear CBM, with a standard deviation across predictions of 5. These values are consistent with Table 11, since the average size is of the same order of magnitude as the metric of Table 11 multiplied by the number of labels for each dataset. The average size is smaller than this quantity, since a single concept can be used in multiple logits.

Table 13. Using CLIP, average size of the prime implicant of each prediction for HCBM and linear CBM, along with the standard deviation of the distribution of prime implicant sizes (mean / std). The standard deviations of these two metrics across repetitions are in brackets when > 10⁻³.

ModelNECWaterbirds-2xView-3CIFAR-10EuroSAT-10RESISC-45
HCBM32.4 / 0.54.2 / 0.8 (0.01/0.01)12.8 / 2.815.2 / 3.1 (0.08/0.02)49.8 / 19.2
54.2 / 0.97.0 / 1.1 (0.1/0.02)19.6 / 3.920.9 / 3.9 (0.1/0.04)73.4 / 25.0
106.5 / 1.611.8 / 1.8 (0.07/0.02)31.0 / 5.432.5 / 5.9 (0.1/0.03)119.7 / 43.6
Linear CBM31.1 / 0.45.2 / 1.1 (0.2/0.04)15.0 / 2.115.1 / 3.6 (0.07/0.03)43.1 / 15.7
51.9 / 1.17.7 / 1.7 (0.2/0.04)20.7 / 4.321.9 / 4.6 (0.03/0.02)62.0 / 15.3
106.3 / 1.414.2 / 2.7 (0.2/0.06)33.2 / 5.735.6 / 6.7 (0.03/0.03)98.5 / 25.0

3. Scope of the main-effect guarantee. Thank you for appreciating the additional Theorem F3. We agree that there is still a gap between the theory and the implemented algorithms. Therefore, we rephrased the claims of the theoretical analysis accordingly in the revised article. Please see our last two comments to Reviewer Lom8 for the details.

Reviewer_Lom8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to combine concept scores in concept bottleneck models (CBMs). The authors argue that a linear model may use correlated but irrelevant concepts to fit a nonlinear relation. They propose HCBM, which uses XGBoost and TreeHFD to build a sparse nonlinear model. The method selects concepts by component variance, adds monotonicity constraints, and uses prime implicants to explain individual predictions. The paper provides theory, tests several image datasets with CLIP and SigLIP, and includes leakage and intervention experiments. It also fits a concept-based model to the class scores of an RTMDet detector on xView.

Critical strengths
  1. The paper gives a concrete reason why nonlinear aggregation may improve concept selection. Appendix B shows how a linear model can use a correlated proxy to fit a nonlinear response. This is a useful explanation of a real failure mode. It gives the method a stronger basis than simply replacing a linear layer with a more flexible model.

  2. The results cover a broad range of settings and show useful gains at low concept budgets. The paper tests several datasets, two vision-language models, and different NEC values. For example, with CLIP and NEC=5, ImageNet accuracy increases from 0.412 for the linear model to 0.605 for HCBM. The intervention results also provide some support for reduced dependence on questionable concepts. On MetaShift, average accuracy changes from 0.848 to 0.831 for HCBM after pruning, compared with 0.850 to 0.781 for the linear model. These results make the method worth further study.

Critical weaknesses
  1. The theory does not yet cover the practical algorithm closely enough. Theorem 2.2 gives a reasonable result for the full Hoeffding decomposition under its assumptions. But Theorem F.1 requires variable subsets of all sizes, while the experiments use only main effects. Appendix C says that including interactions did not change the results at small NEC. This is useful, but it does not show that the main-effect approximation keeps the same variable-selection guarantee. Omitted interactions may still affect concept rankings. The authors should report the decomposition error relative to the original tree model and explain how this error affects selection. A result for the actual main-effect version would be much more convincing.

    The density assumption is another important limit. Appendix E.2 already acknowledges that it fails for RESISC and ImageNet because the concept count exceeds the CLIP embedding dimension. I appreciate this disclosure. Still, good empirical results in these cases do not make the theorem applicable. The main text should state this limit more clearly. It should also explain how the bound in F.1 relates to the empirical variance ranking used in one fitted model.

    Finally, a statistically useful concept score is not necessarily a semantically correct concept. The injected leakage experiments keep the original scores and add degraded copies. They test whether the model rejects redundant proxies. A semantically wrong concept score could still carry extra label information and would not be excluded by this theory. The conclusion already acknowledges concept-task leakage. The broader claims about more faithful and independent concept representations should be kept consistent with that limit, especially because the VLM itself is fixed.

  2. The evaluation does not fully isolate the benefit of Hoeffding decomposition, and some conclusions need a narrower scope. HCBM combines nonlinear fitting, variance-based selection, monotonicity constraints, and retraining. The current results do not show clearly which part produces the gains. EBM is a useful nonlinear control, but its training settings were changed under compute limits for RESISC and CUB. The very low EBM accuracy in these cases makes it hard to use this comparison as strong evidence for the special role of Hoeffding decomposition. On some smaller-class datasets, EBM is better than HCBM.

    The most useful extra experiment would keep the XGBoost model, concept pool, NEC, monotonicity rules, and retraining procedure fixed, and change only the concept selection method. Compare TreeHFD variance with a standard tree importance or another nonlinear selector. Report accuracy, proxy selection rate, and the effect of removing the same number of concepts. A well-tuned nonlinear additive baseline would also help. The introduction should then explain what benefit comes from nonlinear fitting and what extra benefit comes from Hoeffding decomposition.

    The intervention evidence is mixed and should be described that way. On CIFAR-10, HCBM drops from 0.962 to 0.951 after removing irrelevant contributions, while the linear model drops from 0.964 to 0.851. However, the numbers removed are five and thirteen, respectively, as the authors note. On Waterbirds, directly removing background contributions lowers HCBM accuracy from 0.704 to 0.614. Waterbirds on land drops from 0.349 to 0.117. Retraining improves the results, but that is a different intervention. These findings support robustness to some pruning operations, rather than a general guarantee of reliable intervention.

    The xView result is useful as a model of the detector's classification behavior. At NEC=5, agreement with RTMDet is 0.90 for HCBM and 0.67 for the linear model. However, this does not establish detection quality against ground truth. Localization is also outside the explanation, as the authors acknowledge. The authors should either frame this contribution as explaining detected-box classification or add standard detection metrics after replacing the classifier. They should also state whether cross-validation is grouped by original satellite image. The ten-times larger background crops make this detail important. The current description does not rule out shared scene information across folds.

Questions for authors
  1. Which parts of Theorem F.1 apply to the main-effect version used in the experiments? What is the decomposition error relative to XGBoost in these settings? Is there a bound that includes the effect of omitted interactions and supports the empirical concept ranking?

  2. Does the EBM variant use the same concept selection, monotonicity, and retraining steps as HCBM? How were its settings selected and its fit quality checked, especially for RESISC and CUB? This would help judge whether the comparison isolates the decomposition method.

  3. In the xView experiment, are cross-validation folds formed by original satellite image or by individual crop? Is detector training separated from the images used to evaluate HCBM? How are nearby objects with overlapping enlarged crops handled across folds?

  4. Is the intended leakage claim limited to redundant scores that add no label information once the relevant scores are known? If so, can the claims about faithful and independent concept representations be revised to match this scope?

7 comments
AuthorsOpenReview ↗

Rebuttal (weaknesses W1)

Thank you for your detailed review with several relevant suggestions to improve the article. We derived a new theoretical result to include the main-effect setting of HCBM, ran additional experiments for the comparison of XGBoost variable importance and xView cross-validation, and revised the article accordingly with the changes in blue font (new version attached). We hope that these modifications address your concerns, and that you will revise your rating.

W1. The theory does not yet cover the practical algorithm closely enough.

  • Only main effects in TreeHFD. We provide the additional Theorem F3 in Appendix F2 to show that the theoretical guarantees of Theorem F1 still hold when TreeHFD is fit with only main effects. The only additional assumption is that interaction terms in TreeHFD fit with all variable subsets are small, which is consistent with what is obtained in practice for all datasets. The impact in the results of Theorem F3 is that the lower bound on the variance of components of relevant concepts to be ranked above noisy concepts is slightly inflated, compared to Theorem F1. Thank you for this suggestion, which improves the theoretical analysis of HCBM.
  • Assumption of positive density. See the last question in the rebuttal of Reviewer d21A above for the discussion about the positive density assumption. We now refer to this limitation in the main text of the revised article Line 161.
  • Concept-task leakage. HCBM theory indeed supports that the claim of robustness to information leakage is limited to redundant scores that add no label information once the relevant scores are known, that is interconcept leakage. On the other hand, concept-task leakage can still occur with HCBM, which can select semantically wrong concepts, as illustrated in Appendices G4 and G5. This is clearly stated at the beginning of Section 2.2 (Lines 257-261) and in the limitations. However, the sentence Lines 1647-1649 overstate HCBM properties, and we rewrote it. We also highlight that linear CBM select a larger number of irrelevant concepts than HCBM for real datasets, where concept-task leakage can occur, as reported in Appendix G.
AuthorsOpenReview ↗

Rebuttal (weaknesses W2 and questions)

W2. The evaluation does not fully isolate the benefit of Hoeffding decomposition, and some conclusions need a narrower scope.

  • Impact of each component of HCBM. See W1 in the rebuttal of Reviewer d21A for the comparison with the native variable importance of XGBoost, and Q1 for a detailed discussion about the impact of each component of HCBM. Thank you for the relevant suggestion of ablation experiment with XGBoost variable importance.
  • EBM training. EBM is the most widely used algorithm for generalized additive models, and is thus the most relevant competitor of TreeHFD for the nonlinear aggregation. For both XGBoost with TreeHFD and EBM, all default settings are used. However, EBM scales poorly with the number of concepts and classes, since the learning procedure involves each input concept one by one within each of its default 50 000 boosting rounds. For example, a single fit of HCBM on RESISC-45 takes about 10 minutes and around 1 hour for linear CBM, while it was estimated at 200 hours for EBM (after 12 hours of run), as explained in Appendix E2. This makes 8 days of compute only to fit an aggregation of a CBM, and 40 days with 5 repetitions. Running such expensive experiments is not tractable, and this cost shows the inefficiency of EBM in higher dimensions. We reduced the number of boosting rounds to 1000 to have all the EBM experiments for RESISC-45 done within a week, when parallelized over 32 cores. Notice that the poor performance of EBM for RESISC-45 is still obtained with a much higher compute budget than linear CBM and HCBM. On the other hand, TreeHFD takes advantage of the unmatched efficiency of XGBoost combined with the quadratic problem solvers of scipy to obtain fast estimates of the HFD. We finally highlight that EBM has a lower robustness to interconcept leakage than TreeHFD in all cases, even for small input concepts.
  • Interventions. The intervention experiments often show a higher performance of HCBM over linear CBM. However, we better highlight that we mainly focus on pruning interventions in the revised article Line 346.
  • Object detection. Yes, the goal of this case study is to explain an existing black-box detector predictions, not the ground truth. We improve clarity of this point in the contribution paragraph of the introduction. The cross-validation is done by a random partitioning of all extracted crops, and it is indeed a relevant comment that enlarged crops may convey some information of the testing data in the training data. Therefore, we add experiments in Appendix I where the random partitioning is done over the set of images, to enforce that crops extracted from the same image all belong to the same fold of the cross-validation. The results are displayed below in Table 20 for classification, and Table 21 for detection. We see that both the accuracy performance of HCBM and linear CBM slightly decrease, but the strong accuracy improvement of HCBM over linear CBM still holds.

Table 20. For xView with scene-grouped cross-validation, using CLIP, classification accuracy of HCBM with TreeHFD versus sparse linear CBM, in the upper part. Below, proportion (%) of irrelevant concepts with injected leakage selected by HCBM versus sparse linear CBM.

NECLinear CBMHCBM
Accuracy30.6870.867
50.9000.936
100.9530.957
Leaked concepts selected557%0%
1058%3.4%

Table 21. For HCBM and linear CBM, explained variance of each logit and classification accuracy, with respect to predictions of an RTMDet detector for xView with scene-grouped cross-validation.

NECModelExplained variance (Vehicles)Explained variance (Aircraft)Explained variance (Boats)Accuracy
3Linear CBM0.270.390.220.47
HCBM0.680.670.590.87
5Linear CBM0.390.490.390.66
HCBM0.720.700.690.89
10Linear CBM0.630.640.610.87
HCBM0.740.730.740.90

Q1. Theory assumptions. See W1 for the discussion about theoretical assumptions.

Q2. EBM training. Yes, the EBM variant exactly uses HCBM framework, by replacing XGBoost and TreeHFD by EBM. See W2 and Appendix E2 for the discussion about EBM training.

Q3. Object detection with xView. See W2 for the additional experiments about the cross-validation procedure. The accuracy of the detector for xView dataset given in Appendix I is measured on holdout data. Then, CBM are built using all xView images in the cross-validation to keep a pipeline of reasonable complexity. However, we do not state any claim about the performance of CBM with respect to the ground truth, but only about the fidelity of CBM to reproduce the detector predictions.

Q4. Information leakage claim. See W1 about concept-task leakage.

Thank you for the additional analyses and revisions. I appreciate the new main-effect theorem, the XGBoost gain ablation, and the scene-grouped xView evaluation. These additions address important parts of my concerns. However, the practical applicability of the selection guarantees remains insufficiently established, so I am maintaining my current rating at this stage. I would appreciate clarification on the following points.

1. Empirical support for the additional assumption in Theorem F.3.

Theorem F.3 is a substantive extension, but it assumes that every interaction component in the full decomposition of each individual tree is uniformly small. The observation that including interactions does not change accuracy at small NEC does not establish this condition. Could you report quantitative diagnostics in representative settings, including the held-out reconstruction error of the main-effect approximation relative to the fitted XGBoost model, and evidence concerning the magnitudes of the omitted interactions? The reconstruction error would help assess practical applicability, although it would not by itself verify the uniform per-tree assumption. Please also clarify what supports describing the resulting selection threshold as only "slightly inflated", given the dependence of the additional error term on tree depth, the number of trees, and the density bounds.

2. Connection between the theoretical variance ordering and the implemented selection rule.

Theorem F.1 defines variance over all random variables, and the proof of F.3 likewise averages over the training data. In contrast, the algorithm selects concepts using empirical component variances from a particular fitted model. What connects the stated variance ordering to this realized empirical ranking? If a conditional or high-probability guarantee for the implemented ranking is not established, please distinguish the theorem’s conclusion from such a guarantee and qualify the corresponding claims. This was an important part of my original question about how the theory supports practical concept selection.

3. Quantitative results for the matched gain ablation.

The new ablation provides useful evidence for the additional contribution of TreeHFD-based selection. However, Table 10 reports only the proportions of selected injected concepts, while the rebuttal states that the two selection procedures have very similar accuracy. Could you also provide the corresponding accuracy values and variability across repetitions for this comparison? This would help assess whether the improvements in proxy rejection occur at comparable predictive performance and how consistent the smaller differences are.

Correction to my original review: In Table 19, the decreases to 0.614 overall accuracy and 0.117 for waterbirds on land correspond to removing bird-related concepts. Removing background concepts gives 0.641 and 0.221, respectively. My concern about the non-uniform benefits of pruning remains, but the intervention types and numerical results should be matched correctly.

Reviewer_d21AOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Hoeffding Concept Bottleneck Models (HCBM), which leverage Hoeffding Functional Decomposition (HFD) to provide a unique decomposition and then identify concepts with significant contributions while enforcing hierarchical orthogonality among the decomposed components. This promotes concept sparsity and helps mitigate information leakage to some extent. The method further employs prime implicants to identify compact sets of concepts sufficient to explain individual predictions, thereby enhancing interpretability. Building on the recently proposed TreeHFD algorithm, which enables the Hoeffding decomposition of XGBoost models, HCBM captures nonlinear relationships between concept scores and class logits. Experimental results demonstrate that HCBM achieves classification performance comparable to conventional linear CBMs while exhibiting greater robustness to injected interconcept leakage. Overall, the method is supported by a comprehensive theoretical analysis, and the empirical results demonstrate its ability to model nonlinear concept contributions to class predictions.

Critical strengths
  1. An interesting view introducing Hoeffding Functional Decomposition (HFD) into CBMs, providing a good solution to obtain sparse and well-defined concept contributions. This allows CBMs to model more complex, nonlinear concept-to-class relationships while maintaining interpretability. It also introduces a new theoretical perspective for mitigating information leakage in CBMs.

  2. The paper incorporates several additional strategies to improve sparsity and interpretability, such as monotonicity constraints and prime implicants. And also some strategies to meet the constraints of HFD.

  3. The paper provides a relatively comprehensive experimental evaluation, including classification performance and robustness to leakage. The method is also supported by a substantial theoretical analysis, which helps motivate and justify its design.

Critical weaknesses
  1. The paper introduces several important components, particularly nonlinear concept aggregation and HFD, but the experiments do not sufficiently disentangle their individual contributions. It remains unclear how much of the improvement comes from the stronger nonlinear fitting capability and how much can be attributed to HFD. Intuitively, the nonlinear modeling improves the expressiveness of concept-to-class relationships, whereas HFD and concept selection are more directly related to mitigating interconcept leakage. But additional controlled ablations would help clarify the contribution of each component.

  2. The paper demonstrates improved robustness to interconcept leakage mainly through manually constructed injected leakage. It remains unclear how effectively HCBM addresses naturally occurring semantic dependencies or information leakage between related concepts.

  3. The paper presents an extension of HCBM to object detection. However, it primarily focuses on explaining the classification logits of cropped regions, rather than the whole. Although this distinction is partly justified in the conclusion, but I think the one in the intoduction (line 117) needs more clarification.

Questions for authors
  1. The proposed method introduces several components, including nonlinear concept aggregation, HFD-based concept selection, and additional interpretability mechanisms. Could the authors provide more detailed motivation or additional analysis studies to clarify the individual contributions of these components?

  2. Could the authors clarify the applicability of the theoretical guarantees when the assumptions of Theorem 2.1 are not satisfied? Do the uniqueness and irrelevant-concept elimination properties still have any theoretical justification in some failure cases?

Overall, I think introducing nonlinear concept aggregation into CBMs and using HFD to obtain interpretable nonlinear concept contributions is an interesting and promising research direction. The current experimental results provide justification for the overall effectiveness of the proposed approach. If these concerns are adequately addressed, I would be willing to reconsider my rating, as I believe this work has the potential to make a meaningful contribution to the CBM research community.

4 comments
AuthorsOpenReview ↗

Rebuttal (weaknesses)

Thank you for your review and relevant suggestions, and for acknowledging that our article has the potential to make a meaningful contribution to the CBM research community. We address all points below with additional experiments and explanations in the revised version of the article, which is attached. We hope that these changes address your concerns and that you will revise your rating. Let us know if you have further questions.

W1. Ablation experiments to disentangle nonlinear aggregation from HFD. The nonlinear concept aggregation of gradient boosting models and the Hoeffding decomposition (HFD) both contribute to the robustness of HCBM to interconcept leakage, as illustrated in Appendix B. However, these contributions are indeed entangled in the experiments, and we ran an additional ablation experiment to better assess the individual effects, as described in the following paragraph, and added in Appendix G2 in the revised article. Thank you for this suggestion. We also highlight that TreeHFD is required to turn XGBoost into a CBM with an interpretable aggregation, since a gradient boosting model alone is a black-box.

Hence, we conduct the following ablation experiment: the injected leakage experiment of Table 4 is rerun, but with the concept importance of Equation (2) used for the concept selection in HCBM replaced by the default variable importance of XGBoost (gain). Consequently, TreeHFD is not used for the concept selection. The resulting performance is reported in Table 10, and shows that the concept selection of TreeHFD outperforms the native variable importance of XGBoost in most cases. This confirms the efficiency of the HFD to select relevant concepts. Additionally, the performance of the variable importance of XGBoost is much higher than the linear aggregation, which shows that the nonlinear aggregation is key to avoid the selection of irrelevant concepts, which is consistent with the example of Appendix B, where a linear model can select irrelevant concepts by construction to maximize accuracy. Finally, the accuracy of both procedures is very similar, which shows that the nonlinear aggregation is the component to improve accuracy at small NEC, while TreeHFD is the key component to get an interpretable aggregation from XGBoost, and improve robustness to leakage.

Table 10. Using CLIP, proportion (%) of irrelevant concepts with injected leakage selected by HCBM when concept selection uses the native XGBoost importance (gain) instead of the TreeHFD importance of Equation (2).

NECWaterbirds-2xView-3CIFAR-10EuroSAT-10RESISC-45
512% / 0.0%2.4% / 0.0%0.4% / 0.4%0.6% / 0.0%10% / 3.5%
108.0% / 6.0%3.7% / 1.7%3.6% / 1.4%1.1% / 0.7%10% / 11%

W2. Robustness to interconcept leakage in real cases. We mainly focus on manually injected leakage to precisely quantify the robustness of HCBM to leakage, since the ground truth is unknown for real VLM concepts, and interconcept and concept-task leakage are entangled. However, we compare the quality of concept selection between HCBM and linear CBM for EuroSAT and CIFAR-10 in Appendix G4 and G5. For CIFAR-10, linear CBM select 13 concepts that are clearly irrelevant for the corresponding class (among 72 positive concepts), whereas HCBM selects only 5. The CBM models for EuroSAT also show the better concept selection of HCBM. Finally, the higher performance of HCBM for intervention experiments also shows the better concept selection.

W3. Scope of HCBM for object detection. Yes, we clarify this point in the contribution paragraph of the introduction in the revised version of the article. Thank you for this comment.

AuthorsOpenReview ↗

Rebuttal (questions)

Q1. Additional analysis and motivation for each interpretability component of HCBM.

  • Nonlinear concept aggregation vs HFD for concept selection: See W1 above.
  • Normalization of VLM scores: The goal of the min-max normalization of VLM scores is to provide an easy interpretation of the scores with respect to the training data. CLIP scores typically vary between 0.1 and 0.3, and it is therefore hard to know if a score of 0.22 is rather a low or high activation, for example. Also notice that this normalization has no impact on the score aggregation by construction of tree ensembles, as explained in Appendix C.
  • Monotone constraints: The monotonicity constraints do not have an effect on the performance accuracy of HCBM and robustness to interconcept leakage, which remain the same when HCBM pipeline is applied without the monotonicity constraints in the second XGBoost fitting. However, these monotonicity constraints are critical for the explainability properties of HCBM. Indeed, without the monotonicity constraints enforced, most concept components are approximately monotone in the HCBM model, but with local variations that break the monotonicity and are not really significant. This phenomenon quite strongly hurts interpretability for final users. For example, a higher activation of the concept "beak" may slightly decrease the logit of the class ''bird'' around a specific activation strength, because of small estimation errors, which occur because XGBoost has to deal with hundreds of input concepts. With the monotone constraints enforced, the problem disappears. We augment Appendix C with these comments in the revised article.
  • Prime implicants: Prime implicants are an additional mechanism to simplify prediction explanations by focusing on the required concept to obtain the predicted label (See Figures 1 and 3, and Tables 10 and 11 (11 and 12 in revised article) in the Appendix). Prime implicants are computed at in the inference phase and are completely independent of the learning procedure, where the complexity of explanation is controlled by the NEC value.

Q2. Applicability of Theorem 2.1 when assumptions are violated. Yes absolutely, the uniqueness and irrelevant-concept elimination properties still hold for weaker assumptions than those of Theorem 2.1. In particular, we highlight a failure case of these theoretical assumptions at the end of Appendix E: when the number of input concepts is higher than the dimension dL=768d_L=768 of the VLM embeddings, the joint distribution of the concept scores cannot be strictly positive, as assumed in Theorem 2.1 (because of linear relations between concepts). In fact, this assumption of positive density is simple and sufficient to state Theorem 2.1, but is not necessary for the empirical HFD computed by TreeHFD, because the HFD is computed independently for each tree of the ensemble in TreeHFD. Therefore, we only need that the joint density of the concepts involved in a tree is strictly positive. For a tree depth of 6 by default, there are at most 63 concepts used in the splits of a tree (and often much less), which is much smaller than dL=768d_L=768, and the concepts in a tree are thus unlikely to contain a linear relation.

More generally, the strictly positive density assumption is not required to state the Hoeffding decomposition of a given function, but this is still an open research question, which is out of the article scope, and we rely on Chastaing et al. (2012) with the positive assumption. Instead, we provide extensive experiments to show the high efficiency of HCBM to remove irrelevant concepts with interconcept leakage in practice, even with a high number of input concepts where the positive density assumption breaks. We added these comments in Appendix E to highlight the larger scope of Theorem 2.1. Thank you for this question.

Thank you for the clarification and the additional experiments. The new ablation addressed my main concern about the contribution of each part. It shows that treeHFD provides a non-trivial improvement over XGBoost. One direction I would find particularly interesting is to further evaluate the robustness to interconcept leakage claimed in the abstract and formalized after Theorem 2.2. Currently, the injected leakage concepts are nonlinear combinations of a single original concept. Analyzing multi-concept interactions that TreeHFD can model, as mentioned in the abstract, would provide a more convincing interpretable evaluation. I will raise my score to 3.

Thank you for acknowledging that we addressed your main concern, and for raising your score!

We agree that it is interesting to further evaluate the robustness of HCBM to interconcept leakage with concepts containing leakage from multiple relevant concepts, as formalized in Section 2.2. Therefore, we ran the following experiment: we build a new concept with leakage as the product of two initial concepts with noise. The number of injected leakage concepts is the same as the number of initial concepts. The table below shows the high robustness of HCBM to this type of multiple concept leakage, and that HCBM also outperform linear CBM in this context. We will add this table in the final article version (along with datasets with higher number of classes that are longer to run), thank you for this suggestion.

Table: Using CLIP, proportion (%) of concepts with injected leakage selected by linear CBM and HCBM, where each concept with leakage is built as the product of initial relevant concepts with noise. (Each cell: linear CBM / HCBM.)

NECWaterbirds-2xView-3CIFAR-10EuroSAT-10
50.0% / 0.0%59.0% / 4.0%15.2% / 0.0%35.6% / 0.8%
100.0% / 0.0%41.8% / 8.3%17.2% / 1.6%29.5% / 1.7%

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.