- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Proposes HCBM, swaps linear aggregation in CBMs for sparse nonlinear one.CLIP/SigLIP concept scores -> per class XGBoost -> TreeHFD, select concepts by importance,monotone constraints, retrain. prime implicants for per-prediction explanations, plus a result that irrelevant concepts get zero components (under full Hoeffding decomp). Exps on natural+remote sensing, injected leakage, interventions in appendix, xView w/ RTMDet logits (detector faithfulness, not localizaton)
Critical strengths
Big gains at small budgets, and concepts/features fixed so its really the aggregation. CLIP RESISC NEC=3 0.673->0.816, ImageNet NEC=5 0.412->0.605
not just qualitative plots — MetaShift deletion HCBM 0.848->0.831 vs linear 0.850->0.781. nice, tho doesn't show concepts are semantically right/causal
Critical weaknesses
prime implicant code can return non-implicants. hcbm.py L414-423 sums h_values over ALL concepts (minus deleted ones) when checking removal,so concepts never in the candidate set count as fixed.
counterexample: u_j=1[z_j>=0.5], Δ=-4+3u1+2.1u2+2u3. z=(1,1,1) gives 3.1, candidates {1,2}, code returns {1}. but Δ(1,0,0)=-1 → flips. reproduced it. so sufficiency fails not just minimality. linear CBM code looks the same so both length stats are suspect
also L348-369 uses 1000pt grid for the min, can miss narrow gaps btw splits. got a 2nd invalid case from that too
Theory doesnt really transfer. Thm 2.2/F.1 need full decomp (all orders, density bounds), exps use main effects only — truncated interactions can leak into correlated proxies. E.2 admits RESISC/ImageNet break the density assumption, fair, but then cant use the thms there.
eq(2) is relative variance per class but code (L257-258,L150-168) pools raw variance acros classes?? which one was used
Questions for authors
Tables 10-11 / examples made with the attached compute_prime_implicants? pls fix, use exact mins at breakpoints, report sufficiency checks+new lengths for both models. full |S(X)| per prediction would help
w/ interaction_order=1 what of F.1 still holds,extra additivity assumption? narrowing claims is fine. also eq(2) vs code
xView CV is random KFold on crops, no grouping by image/region. same for Table 2? overlapping crops across folds → want grouped results
3 comments
Thank you for the detailed response and revisions. The correction to Equation (2) resolves my concern about the mismatch between the stated importance measure and the implementation. I also appreciate the scene-grouped xView experiments in Tables 20–21, which address my concern about crops from the same source image appearing across folds. The additional Theorem F.3 is a meaningful extension to the main-effect setting.
My remaining concerns are primarily about verifying the corrected prime-implicant results and clarifying the scope of the guarantees.
1. Verification of the corrected prime-implicant implementation.
The cutpoint-based minimum computation described in the revised Appendix C is an appropriate approach. However, my original concern was that the returned sets could fail to be sufficient, rather than only having underestimated sizes. Updated average sizes alone do not establish that this issue is resolved.
Could you provide the corrected implementation or a minimal patch for both HCBM and the affected linear-baseline routine? The code archive available in the materials I checked appears to contain the original implementation, so this may simply be a version mismatch.
Please also provide regression checks for the counterexample in my review and a narrow-interval case missed by the uniform grid. For a representative evaluation set, please report the number of returned explanations checked and any sufficiency violations, using exact worst-case margins against every competing class while allowing all concepts outside the returned set to vary. This would directly verify the property that was affected by the bug.
2. Full explanation size per prediction.
Tables 11–12 continue to report the average number of implicant concepts per logit. This differs from the total number of distinct concepts in the sufficient explanation, , which I requested in the original review. Could you additionally report the full-set size per prediction, with a mean and distribution summaries? This would help assess the compactness of the complete explanation, particularly in the multiclass settings.
3. Scope of the main-effect guarantee.
I appreciate Theorem F.3 and consider it substantive progress. Its additional assumption concerns uniformly small interaction components in each individual tree, which is stronger than observing little change in predictive performance when interactions are omitted. I also share the concern raised in the discussion about distinguishing variance ordering averaged over the training data from the empirical ranking of one fitted model. Representative quantitative evidence would be useful; alternatively, the corresponding claims should clearly state their conditional scope and distinguish them from a guarantee for the implemented empirical ranking.
For now, I am maintaining my rating of 2 (Weak rejection), pending clarification of these remaining points. I recognize the empirical improvements at small concept budgets and the substantial work done in the revision, and I remain open to increasing my rating once the corrected explanation procedure is verifiable and the claims are appropriately scoped.
Thank you for your quick reply to our rebuttal, and for highlighting that we addressed many of your initial concerns. We address below the remaining points.
1. Verification of the corrected prime-implicant implementation. Our new implementation of prime implicants now provides a list of concepts for each HCBM prediction which satisfies the property of sufficiency, and no strict subset of the list also satisfies it, because we compute exact minimum of main effect differences. We checked that our implementation now outputs valid prime implicants for the counterexample of your initial review and for cases with narrow intervals.
We indeed did not provide the updated code in the first revision, but we have now updated the attached Supplementary material with the new code version.
We specify two details: when the two top logits are tied for a given prediction (very rare case), no implicant exists by definition, and we thus consider that no concept can be dropped by convention, and all model concepts are returned. We also corrected a typo in the implementation of prime implicants for linear CBM in the specific case of two classes (with only one logit, as in Waterbirds dataset), and updated the metrics.
2. Full explanation size per prediction. In Table 13 of Appendix G3, we now report the average size of prime implicants for each prediction, along with the standard deviation of the size distribution across all predictions. Thank you for this suggestion, since this metric seems easier to read than our initial one. The results confirm the high efficiency of prime implicants to strongly reduce the number of required concepts to explain a given prediction. For example, for CIFAR-10 with NEC at 10, 100 concepts are involved in the CBM, and prime implicants have an average size of 31 for HCBM and 33 for linear CBM, with a standard deviation across predictions of 5. These values are consistent with Table 11, since the average size is of the same order of magnitude as the metric of Table 11 multiplied by the number of labels for each dataset. The average size is smaller than this quantity, since a single concept can be used in multiple logits.
Table 13. Using CLIP, average size of the prime implicant of each prediction for HCBM and linear CBM, along with the standard deviation of the distribution of prime implicant sizes (mean / std). The standard deviations of these two metrics across repetitions are in brackets when > 10⁻³.
| Model | NEC | Waterbirds-2 | xView-3 | CIFAR-10 | EuroSAT-10 | RESISC-45 |
|---|---|---|---|---|---|---|
| HCBM | 3 | 2.4 / 0.5 | 4.2 / 0.8 (0.01/0.01) | 12.8 / 2.8 | 15.2 / 3.1 (0.08/0.02) | 49.8 / 19.2 |
| 5 | 4.2 / 0.9 | 7.0 / 1.1 (0.1/0.02) | 19.6 / 3.9 | 20.9 / 3.9 (0.1/0.04) | 73.4 / 25.0 | |
| 10 | 6.5 / 1.6 | 11.8 / 1.8 (0.07/0.02) | 31.0 / 5.4 | 32.5 / 5.9 (0.1/0.03) | 119.7 / 43.6 | |
| Linear CBM | 3 | 1.1 / 0.4 | 5.2 / 1.1 (0.2/0.04) | 15.0 / 2.1 | 15.1 / 3.6 (0.07/0.03) | 43.1 / 15.7 |
| 5 | 1.9 / 1.1 | 7.7 / 1.7 (0.2/0.04) | 20.7 / 4.3 | 21.9 / 4.6 (0.03/0.02) | 62.0 / 15.3 | |
| 10 | 6.3 / 1.4 | 14.2 / 2.7 (0.2/0.06) | 33.2 / 5.7 | 35.6 / 6.7 (0.03/0.03) | 98.5 / 25.0 |
3. Scope of the main-effect guarantee. Thank you for appreciating the additional Theorem F3. We agree that there is still a gap between the theory and the implemented algorithms. Therefore, we rephrased the claims of the theoretical analysis accordingly in the revised article. Please see our last two comments to Reviewer Lom8 for the details.
Rebuttal
Thank you for the thorough verification of our code, and the identified bugs and typos. We corrected all of them and reran experiments to show that they all have no impact on performance metrics, except the post-treatment of prime implicants which slightly underestimated their size, but leaves the claims about prime implicants efficiency untouched. We also added a new theoretical result and additional experiments for xView. We hope that these improvements address your concerns, and that you will revise your score. (Changes are in blue font in the attached revised article.)
W1. Code verification for prime implicants. There is indeed a bug in the code for the post-treatment of prime implicants. We fixed the code and rerun experiments of Tables 10 and 11 (now 11 and 12 in the revised article), to show that the sizes of prime implicants are slightly higher for both linear CBM and HCBM, but are still similar to the original values. Therefore, the claim that prime implicants efficiently simplify HCBM explanations still hold. Thank you for pointing this out.
Additionally, we also added the exact computation of minimum of the difference of TreeHFD main effects. Since TreeHFD main effects are piecewise constant, we simply need to retrieve all cut points of all trees for each concept, build an adaptive grid with one point between two consecutive cut points, and take the minimum over this set of points for each concept. The results of this improvement combined with the bug fix are displayed in the updated Tables 11 and 12. The impact of computing exact minimum compared to the minimum approximation with the regular grid is probably negligible for the prime implicant sizes, as opposed to the bug fix.
W2. Code mismatch for concept importance. Indeed, there is a typo in Equation (2), where the right-hand side should be only α, as implemented in the code and used in all the theoretical analysis of Appendix F2. Thank you for pointing this out. We thus corrected Equation (2) in the revised article. Also notice that this normalization has almost no effect in practice, since all logits have close variance for balanced classification. We tested to add this normalization in the code, and then reproduced the accuracy performance of Table 4. However, this normalization should not be included, because if a given class has a very small number of relevant concepts, noisy ones may be selected in specific cases.
W3. Theory assumptions. We provide the additional Theorem F3 in Appendix F2 to show that the theoretical guarantees of Theorem F1 still holds when TreeHFD is fit with only main effects. See W1 in the rebuttal of Reviewer Lom8 for the discussion about this new result. See also the last question in the rebuttal of Reviewer d21A above for the discussion about the positive density assumption.
Q1. Implementation of prime implicants. See W1.
Q2. Theory assumptions. See W3.
Q3. Object detection. See W2 about object detection in the rebuttal of Reviewer Lom8.