- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
model predictions. The method uses SVD to extract low-dimensional concept subspaces from a small number of positive examples, followed by a hierarchical Shapley-based approach to attribute prediction changes to concepts and individual modules. Experiments on CIFAR-100, CUB-200-2011, and ImageNet-1k show competitive concept identification performance with substantially lower fitting costs than concept bottleneck methods.
Critical strengths
S1. ViTLens achieves competitive concept identification performance with low computational cost and without modifying the original model.
S2. 实验比较全面,概念抑制和灾难性遗忘两个应用也为理解 ViT 的内部行为提供了有价值的发现。
Critical weaknesses
W1. The method extracts concept subspaces using SVD with only 10 positive examples. These examples may share backgrounds or class-related features. Could the authors further verify that the extracted subspaces capture the target concepts rather than other correlated features?
W2. The hierarchical Shapley approximation is validated only on the last 3 layers with a small number of concepts, leaving its reliability across the full ViT insufficiently established.
W3. The paper emphasizes computational efficiency, but ViTLens does not improve fitting speed over Linear Probe on some datasets. Its efficiency advantage is mainly demonstrated against CBM-based methods.
Questions for authors
Q1. Since the Z-score normalization depends on the concept set, would adding new concepts affect the detection scores of existing concepts? Would the thresholds need to be recalibrated?
Q2. Table F.2 shows that smaller subspace ranks sometimes perform better. Why is rank 8 used for all concepts and datasets? Could an adaptive rank selection improve performance?
4 comments
Response to Reviewer K327
Q1: Since the Z-score normalization depends on the concept set, would adding new concepts affect the detection scores of existing concepts? Would the thresholds need to be recalibrated?
We thank the reviewer for raising this. In short, adding new concepts leaves the detection of existing concepts essentially unchanged, and the threshold does not need to be recalibrated.
This is because each subspace is fitted only on the positive exemplars of , so adding concepts never changes an existing subspace or its energy in Eq. (5). The concept set only enters through the per-image mean and standard deviation , so adding concepts maps every existing score to with . Thus, at any single layer, the ranking of concepts within an image cannot change, and only the position of the scores relative to can move.
To measure this, we start from a random 50% of the concepts on ViT-B/16 and add 25%, 50% and 100% of the remaining ones. is selected once on the validation split with the initial set and then frozen, and we score only the evaluated concepts of the initial set (about 300, 350 and 2,080 of the 608, 698 and 4,167 in Table 1) at all four Table 1 sites.
| Max |ΔF1| from re-selecting | ΔF1 at frozen | Predictions changed | |
|---|---|---|---|
| CUB-200-2011 | 0.30 | +0.05 to +0.47 | ≤ 0.09% |
| CIFAR-100 | 0.08 | −0.06 to +0.46 | ≤ 0.10% |
| ImageNet-1k | 0.18 | −0.03 to +0.29 | ≤ 0.01% |
At the frozen , macro-F1 changes by less than 0.5 for all 12 dataset × site combinations, and re-selecting gains at most 0.30. Moreover, the ranking of concepts within each image is exactly preserved at Last Res, and almost exactly at Res (median per-image Spearman correlation above 0.999). Thus, no recalibration is needed.
Q2: Table F.2 shows that smaller subspace ranks sometimes perform better. Why is rank 8 used for all concepts and datasets? Could an adaptive rank selection improve performance?
We agree that a smaller rank sometimes performs better in Table F.2. However, the gain is marginal: selecting the best rank among for each site and dataset on the test set improves macro-F1 over by only 0.20 on average, and is already the best rank in 7 of the 12 cases. Thus, a fixed is sufficient.
Nevertheless, adaptive rank selection is an interesting direction. Motivated by the hierarchical organization of concept subspaces in Appendix I.2, each concept could be modelled by a flag of nested subspaces, i.e. a point on the flag manifold [4], which comes at no extra cost as the singular vectors in Eq. (2) are already nested [5]. Weighting the energy of each nested block in Eq. (5) would then permit concept- and layer-dependent effective ranks, with the current formulation as a special case. We leave this as future work.
References
[1] Piatetsky-Shapiro, Gregory, and Sam Steingold. "Measuring lift quality in database marketing." ACM SIGKDD Explorations Newsletter 2(2), 2000.
[2] Postels, Janis, et al. "On the practicality of deterministic epistemic uncertainty." ICML 2022.
[3] Srivastava, Divyansh, Ge Yan, and Tsui-Wei Weng. "VLG-CBM: Training concept bottleneck models with vision-language guidance." NeurIPS 2024.
[4] Ye, Ke, Ken Sze-Wai Wong, and Lek-Heng Lim. "Optimization on flag manifolds." Mathematical Programming 194, 2022.
[5] Pennec, Xavier. "Barycentric subspace analysis on manifolds." Annals of Statistics 46(6A), 2018.
Details of the shared concept analysis (W1)
Below we give the full setup of the shared concept analysis on CIFAR-100 reported in W1.
Pair selection. Each CIFAR-100 class has 100 test images. We keep a concept–class pair when (i) class has at least 10 positives and at least 10 negatives of among its test images, (ii) satisfies (i) in at least 2 classes, (iii) the median detection confidence of the pseudo-labeler [3] on the pair is at least 0.20, and (iv) has positives in the training split (at least 2 images per class) in at least 2 classes other than , so that the subspace of can be fitted on other classes and tested on a class it has never seen. Condition (iv) reduces the 61 shared pairs to 49 (16 concepts, 36 classes), with 12 to 89 positives per pair. The subspaces are fitted on the training split, and the pairs are scored on the test split.
Metric. For each pair with test images of class , labels , positives and base rate , we report the Lift metric [1, 2]
where Top- denotes the images of class with the highest . We use the Lift metric as it requires no threshold, and chance is exactly for every pair regardless of its base rate. The mean and s.e. are taken over the 49 pairs.
Settings. All settings use ViT-B/16. The rank (), the number of fitting images (10) and the scoring are kept identical, and only the images used to fit the subspace are changed:
- ViTLens: 10 positives of , as in Table 1.
- ViTLens-Decorrelated: 10 positives of from other classes only, so class is never seen.
- Class-only: 10 images of class with the concept labels ignored, which gives a rank-8 subspace spanned by the class.
- VLG-CBM [3]: the supervised concept layer trained on the full labelled training set, with its scores z-scored across concepts as for ViTLens, scored on the same pairs.
[1] Piatetsky-Shapiro, Gregory, and Sam Steingold. "Measuring lift quality in database marketing." ACM SIGKDD Explorations Newsletter 2(2), 2000.
[2] Postels, Janis, et al. "On the practicality of deterministic epistemic uncertainty." ICML 2022.
[3] Srivastava, Divyansh, Ge Yan, and Tsui-Wei Weng. "VLG-CBM: Training concept bottleneck models with vision-language guidance." NeurIPS 2024.
Follow-up on Rebuttal
Dear Reviewer K327,
We would like to kindly follow up on our rebuttal and the additional experiments. We would appreciate any further feedback or remaining concerns we could address before the discussion closes.
Thank you again for your time and consideration.
Response to Reviewer K327
We thank the reviewer for the thoughtful feedback, and for recognizing that ViTLens achieves competitive concept identification with low computational cost and without modifying the original model. For S2, "实验比较全面,概念抑制和灾难性遗忘两个应用也为理解 ViT 的内部行为提供了有价值的发现。", we translated it with an AI tool, and it means "The experimental comparison is comprehensive, and the two applications of concept suppression and catastrophic forgetting also provide valuable findings for understanding the internal behavior of ViTs." We are grateful for this positive assessment, and we will address each concern of the reviewer below.
W1: The method extracts concept subspaces using SVD with only 10 positive examples. These examples may share backgrounds or class-related features. Could the authors further verify that the extracted subspaces capture the target concepts rather than other correlated features?
For background-related features, we apply the subspaces fitted on the [CLS] token to the [PATCH] tokens as shown in Fig. B.1. If a subspace mainly captured the shared background, the energy would concentrate on the background patches. Instead, it concentrates on the object, e.g. "towering height" on the mountain peak and "tower" on the water tower. However, there are failure cases where the energy focuses on the wrong object, e.g. "crucifix" on the altarpiece which we show in Fig. B.2.
For class-related features, we further fit the subspace of concept c only on positives of c from other classes (Decorrelated), and score it on the test images of a class k it has never seen. As a control, Class-only fits the subspace on 10 images of class k with the concept labels ignored, which gives a rank-8 subspace spanned by the class features. The rank (r=8), the number of fitting images (10) and the scoring are kept identical. We report the Lift metric [1, 2], the fraction of positives among the top-nc,k scored images of class k divided by the base rate, so chance is exactly 1 for every pair. We filter CIFAR-100 for concept–class pairs (c,k) where class k has at least 10 positives and 10 negatives of c among its test images, and c is shared by at least 2 such classes. The full details of this analysis are given in a separate reply below.
By fitting on other classes only, Decorrelated is +0.395 above Class-only and on par with VLG-CBM trained on the full labelled training set, while Class-only stays at chance. Thus, the subspace captures the concept shared across classes rather than the features of the class it is fitted on.
W2: The hierarchical Shapley approximation is validated only on the last 3 layers with a small number of concepts, leaving its reliability across the full ViT insufficiently established.
Validating the full hierarchy across all 12 layers is infeasible, as the exact Shapley value of the joint game over 5 concepts and 12 layers requires 260≈1.15×1018 coalition evaluations per image (Appendix D.3). The last 3 layers with 5 concepts already cost 215=32,768 coalition evaluations per image, and each additional layer multiplies this by 25=32, so we restrict the test to this setting to keep exact enumeration tractable over the 400 test images. We choose the last 3 layers as they directly produce the output, and a removal at each of them propagates through the following ones.
W3: The paper emphasizes computational efficiency, but ViTLens does not improve fitting speed over Linear Probe on some datasets. Its efficiency advantage is mainly demonstrated against CBM-based methods.
We agree that ViTLens and Linear Probe have a similar fitting time (Table F.4), but at the same cost ViTLens outperforms Linear Probe by 20.01 macro-F1 over the FFN, Attn and Res sites and by 8.87 at Last Res (Table 1).
Moreover, Linear Probe is a single classifier shared by all concepts and must be retrained to add a new one, whereas ViTLens fits each concept independently and adds it without modifying the existing subspaces.
Finally, the cost of Linear Probe grows with the number of optimization steps, and it becomes hard to train with a large concept set such as the 4,167 concepts of ImageNet-1k, whereas ViTLens fits each concept with one SVD of its 10 exemplars at a fixed cost.