acceptodds
Withdrawn from ICLR 2027

VisionLogic: Discovering and Grounding Decision-Relevant Visual Concepts

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Concept-based explanations help users understand vision models through recognizable visual patterns. However, existing methods often rely on correlational signals without directly validating which image cues support prediction-relevant internal features. To this end, we introduce VisionLogic, a post-hoc framework that grounds these features in visual concepts through intervention-based validation. VisionLogic first identifies compact sets of features whose contributions reproduce the model's original prediction. It then represents their activation states as predicates using class-specific thresholds. An iterative refinement procedure grounds these predicates in visual regions through ablation tests. A region is accepted when its removal deactivates the corresponding predicate, linking the feature's numerical role to visual evidence in the input. The same predicates allow us to examine how features are activated, selected, and reused across images and classes. Across CNNs and vision transformers on ImageNet-1k, we find that only a few features are selected to explain each prediction, and frequently active features are not always selected. In a large-scale human evaluation with 465 participants, VisionLogic significantly improves participants' understanding of model behavior over established methods ACE and CRAFT.

Primary area:
interpretability and explainable AI
Keywords:
vision interpretability, concept-based explanations, feature discovery, intervention-based grounding, global feature analysis, human-centered evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_shkBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VisionLogic, a framework designed to explain vision models by grounding decision-relevant visual concepts. VisionLogic identifies compact feature sets that replicate the model's predictions, representing these features as predicates based on class-specific thresholds, and grounds these predicates in visual regions using an iterative procedure. The framework is assessed across different models and shows improvements in understanding model behavior through human evaluation compared to established methods.

Critical strengths
  • This paper provides a large-scale (465) human evaluation, which measures humans’ ability to predict model outputs on unseen images with the help of explanations.
  • The experiments are conducted on ResNet, ViT, ConvNeXt, and Swin, and demonstrate better results than baselines in human studies.
  • Some interesting analysis about feature use and selection in classification models is carried out with the proposed explanation method.
Critical weaknesses
  • Only human user studies are used to evaluate the proposed method compared with baselines and other concept explanation methods.
Questions for authors
  • Is there any quantitative metric for the evaluation? Although user studies show that the explanation is indeed helpful for humans to better understand the model, I think that just using user studies is not sufficient for evaluating the faithfulness of the method, since human responses are usually noisy.
  • The threshold is decided across the source images predicted as class c. So how does the number of source images influence the threshold, and the further analysis results and visualization results?
  • How often do predicates activate outside their associated class?
Reviewer_DT7fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

VisionLogic is a post-hoc explanation method for frozen image classifiers that aims to identify decision-relevant features, represent them as Boolean predicates, and ground them in image regions. It first selects penultimate-layer features according to their contribution to the predicted class logit until the classifier head reproduces the original prediction. It then defines class-specific activation thresholds for the selected features and uses Grad-CAM, Score-CAM, and region blurring to identify boxes whose removal causes a predicate to become inactive. The paper evaluates the method on four ImageNet classifiers, reporting feature-selection statistics, qualitative concept examples, and a human study with 465 retained participants. The results suggest that relatively few features can reproduce many predictions, that some features are reused across classes, and that VisionLogic can improve human utility in at least one of the evaluated scenarios. However, the evidence that the grounded regions consistently represent meaningful visual concepts, and that the method provides reliable human benefits across tasks, remains incomplete.

Critical strengths
  1. Clear and well-specified method. The pipeline is described in detail, including feature ordering, tie-breaking, blurring, and the cutoff schedule. The appendices provide enough information to understand and potentially reproduce the procedure.

  2. Useful sanity checks and statistical reporting. The held-out feature-selection results provide evidence that the selected features generally have corresponding predicates and satisfy the specified thresholds. The authors also use non-parametric statistical tests with multiple-comparison correction and report the non-significant results rather than presenting only favorable findings.

  3. Evaluation across multiple architectures and human tasks. The experiments cover four ImageNet models and include a human-centered evaluation with 465 retained participants. The study is a useful attempt to assess explanations through their effect on human decision-making rather than relying exclusively on model-side metrics.

Critical weaknesses
  1. The grounding evaluation does not yet establish that the accepted regions capture meaningful visual evidence. A region is accepted when blurring it pushes a feature below a threshold defined using its minimum activation among selected source examples. This establishes a form of necessity under the chosen intervention, but does not show that the region corresponds to the semantic concept represented by the feature. The random-box control is particularly relevant: randomly placed, area-matched boxes deactivate 92.8% of predicates for ViT and 81.4% for ResNet, compared with 99.3% and 89.9% for guided search. The guided method performs better, but the improvement is relatively modest. It remains unclear how much of the observed deactivation reflects meaningful localization versus the effects of blurring and thresholding. The accepted regions are also often large, with median image occupancy of 24% for ViT and 46% for ResNet. There is no sufficiency test, specificity test, or analysis of the activation drop relative to the threshold margin. The grounding experiment is also limited to one seed and 100 images, without reported uncertainty estimates. These limitations directly affect the paper's central claim that it moves beyond correlational explanations.

  2. The human-study results do not establish consistent gains across scenarios. The clearest evidence for VisionLogic appears in Husky vs. Wolf. In Otter vs. Beaver, VisionLogic does not significantly outperform the baseline or control, with reported p-values of 0.156 and 0.146, respectively. The baseline is already close to 89%, making the scope for improvement limited, and the paper does not sufficiently establish why this scenario demonstrates a non-obvious strategy. In Kit Fox vs. Red Fox, the omnibus test is not significant at the 0.05 level (p = 0.058), and VisionLogic's reported utility is below the baseline. Overall, the study supports a more limited claim than the abstract and conclusion suggest.

  3. The human study does not isolate which component of VisionLogic drives any observed improvement. VisionLogic presents per-image boxes, whereas ACE and CRAFT are global concept-based methods adapted to the study's presentation format. Without a plain Grad-CAM-box baseline, a VisionLogic variant without the ablation-based acceptance filter, or a random-box condition, it is difficult to determine whether the gains come from the proposed grounding procedure, the visual format, or other aspects of the experimental setup.

  4. The comparison with ACE and CRAFT needs further explanation. The reported utilities differ substantially from those in the original CRAFT study. For Husky vs. Wolf, the original study reports utility values of 1.53 for CRAFT and 1.15 for ACE, whereas this paper reports 0.89 and 1.01, respectively. Baseline accuracies also differ between the studies. These differences do not by themselves establish a problem with the experiments, but the description that the methods were rerun on adapted datasets does not explain what changed or how the results should be compared with the published findings.

  5. The feature and concept claims need stronger validation. The observations about feature reuse and the small number of features needed to reproduce predictions are interesting, but they may partly reflect the geometry of the final 1000-class linear head and the chosen contribution ordering. The paper does not examine sensitivity to the threshold rule or feature ordering, or compare against a suitable null model. Similarly, the examples of concepts such as fox ears and bird beaks are qualitative. A quantitative evaluation against part annotations or another independent concept-grounding criterion would make these claims more convincing.

  6. The distinction from related approaches could be clearer. The paper should more directly explain how VisionLogic differs from methods such as CRP, NeSyFOLD/NeSyFOLD-G, and vision sparse autoencoders, particularly regarding the use of internal features, predicate construction, and visual grounding. This would help establish the specific methodological contribution beyond combining existing feature attribution and perturbation techniques.

Questions for authors
  1. Which classifier was used in each human-study scenario, and how were the Husky/Wolf and Otter/Beaver classifiers trained? What evidence supports the claim that the Otter/Beaver task requires a non-obvious strategy, given the high baseline accuracy?
  2. What exactly was adapted from the original CRAFT and Colin et al. evaluation protocols? Please clarify any changes to the image sets, models, explanation displays, ACE/CRAFT hyperparameters, and evaluation procedures. How should the differences in utility and baseline accuracy between the original and current studies be interpreted?
  3. How many boxes or predicates were displayed for each image? How many participants were recruited and retained in each condition, and how were the exclusion criteria applied? Please clarify the differences in retained group sizes across conditions.
  4. Given that random area-matched boxes already deactivate a large proportion of predicates, what evidence indicates that the accepted regions capture meaningful visual evidence rather than blur and threshold effects? Can the authors report the activation drop relative to the threshold margin and evaluate sufficiency by retaining the proposed region while blurring the rest of the image?
  5. How sensitive are the feature-selection and feature-reuse results to the threshold rule and contribution ordering? Do the main findings persist under random feature ordering or an appropriate shuffled-head control?
  6. How were the qualitative examples in Figures 2–4 and 6 selected? Can the semantic interpretations be validated quantitatively, for example against object-part annotations or another independent grounding measure?
Reviewer_o9xzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces VisionLogic, a post-hoc explainability technique which extracts feature attributions as predicates, and visualizes them by identifying the relevant region through a gradient-based approach followed by occlusion and intersecting it with a foreground segmentation obtained by an off-the-shelf segmented model. The proposed method is applied on both CNNs and ViT architectures on Imagenet-1k, tested with a large scale human evaluation (500+ participants) against ACE and CRAFT showing improvements, and analyzed both qualitatively and quantitatively.

Critical strengths

None

Critical weaknesses
  1. Methodology

Considering the document, Section 3 contains some inconsistencies, i.e., some terms are used but never explained, the margin metric is introduced but it is unclear for what is it used, feature dimensions are never discussed, and some design . Regarding the method itself, it has some unaddressed limitations: (a) faithfulness of the explanations was not tested, and the intersection between bounding boxes and foreground segmentations assumes that the foreground object is the most relevant portion; in situations where a model uses only background information to classify (e.g., the example of [1] where the snowy background is identified as the most discriminative part for predicting the husky class), the intersection may not consider the most meaningful part, showing misleading overlays; (b) the extracted concepts must have spatial meaning and are subject to saliency methods’ limitations (e.g., if the class-discriminative concept was something abstract or a specific body part of a living being, it is unclear whether the method would be able to highlight it); (c) the method considers only features of the final linear classification head as concepts, reducing the scope of explanations;

  1. Limited findings

The results provided in section 4 are coherent with contributions 3 and 4, although they provide limited advancements. Regarding the human evaluation, the utility score assesses the understandability of explanations, but no other metric was evaluated, and the foreground segmentation overlay was not used in the experiments as stated in the Appendix. Regarding the analysis of feature activation, selection, and reusability, the main findings reported in the conclusions are that subsets of features are able to reproduce a prediction even when more features are active, and that features are reused across multiple classes. These highlight the superposition problem, a well known issue in literature which is being studied in Mechanistic Interpretability and has already been addressed by SAEs for both language [2] and vision models [3].

References

[1] Ribeiro, Marco Tulio, Sameer Singh, and Carlos Guestrin. ""Why should i trust you?" Explaining the predictions of any classifier." Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016.

[2] Huben, Robert, et al. "Sparse autoencoders find highly interpretable features in language models." International Conference on Learning Representations. Vol. 2024. 2024.

[3] Pach, M., Karthik, S., Bouniot, Q., Belongie, S., & Akata, Z. (2026). Sparse autoencoders learn monosemantic features in vision-language models. Advances in Neural Information Processing Systems, 38, 95706-95742.

Questions for authors

1 - Have the authors tested methods other than blurring for occluding the relevant regions?

2 - Could the authors better clarify the motivations behind the foreground assisted visualization step given the observations in 1a and considering its absence on the human evaluation experiment?

3 - By looking at the qualitative examples, different predicates seem to identify similar concepts. Could the authors quantify how different are the extracted concept heatmaps given a sample image and all its predicates?

4 - Did the human experiments use the free-form segments of ACE or were they reduced to rectangular bounding boxes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.