acceptodds
Withdrawn from ICLR 2027

Supervised Concept Discovery for Interpreting Vision Representations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision foundation models encode rich visual information, yet their representations do not directly expose human-interpretable factors. Concept discovery seeks to reveal these factors by decomposing pretrained representations into latent directions. However, reconstruction and sparsity objectives provide limited explicit guidance toward semantically coherent factors. We investigate class labels as a source of such guidance and introduce label impurity, an objective for guiding factor learning. Minimizing label impurity encourages each factor to exhibit consistent activation states among images of the same class, while allowing factors to be shared across classes. We derive an unbiased minibatch estimator for this objective to support stable stochastic optimization. Empirical results show that our approach discovers interpretable and visually coherent factors across diverse visual recognition datasets.

Primary area:
interpretability and explainable AI
Keywords:
concept extraction, interpretability, sparse autoencoders, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_o39ROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a label impurity regularizer for supervised concept discovery using sparse autoencoders (SAEs). The objective encourages consistent factor activations within each class, and an unbiased minibatch estimator is derived for efficient optimization. The authors further adopt binary activations through HTAF. Experiments on ImageNet-1K, CUB-200, and Places365 evaluate classification accuracy, semantic alignment, and concept coherence.

Critical strengths

1.Simple and efficient formulation. The proposed objective incorporates class supervision without requiring concept-level annotations. The unbiased minibatch estimator makes the regularizer practical for SAE training.

2.Improvements from label supervision. SAE+LI consistently improves classification accuracy over vanilla SAE across three datasets and two CLIP backbones (Table 3). The qualitative examples also suggest that some learned factors capture recognizable visual attributes.

Critical weaknesses
  1. Limited methodological novelty. The method mainly adds a class-conditional activation consistency regularizer to an existing SAE framework, while HTAF is adopted from prior work. The objective is closely related to minimizing within-class activation variance. Its advantage over existing supervised representation learning approaches is not sufficiently established, and the absence of an HTAF-only ablation makes it difficult to isolate the contributions.

  2. Insufficient evidence for improved interpretability. Table 4 primarily relies on CLIP-based semantic similarity, which does not directly measure whether the discovered factors are human-interpretable. The qualitative results are also selectively presented. Moreover, the full method underperforms SAE+LI in classification accuracy across all six settings in Table 3. Stronger independent evaluation is needed to justify the claimed improvement in concept quality.

Questions for authors
  1. Eq. (3) also favors factors that are constant within each class. What ensures that the model learns meaningful visual concepts rather than simply class-specific representations?

  2. Table 4 relies on CLIP similarity, while Section 3.4 uses class labels to name concepts on CUB-200. Can the authors validate the improvements using independent concept annotations, such as CUB-200 attributes?

  3. Tables 3–4 omit the SAE+HTAF baseline without label impurity, and Appendix D does not report concept quality for SAE+CE. How do these baselines compare with the proposed method on concept quality?

Reviewer_pBuJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies supervised concept discovery from frozen pretrained visual representations (CLIP encoders in the experiments), which are decomposed using sparse autoencoders (SAEs). The authors hypothesize that class labels can guide the discovery of semantically coherent factors, including factors shared across classes. They introduce a label-impurity objective that encourages each factor’s class-conditional mean activation to approach zero or one, promoting within-class activation consistency. They derive an unbiased degree-two U-statistic estimator that can be computed efficiently from minibatches. They also use a differentiable approximation to the Heaviside function during training, yielding binary factor activations at inference, and combine reconstruction, sparsity, and label-impurity terms in the training objective. For semantic naming, the paper matches decoder directions to vocabulary terms for general object and scene datasets, and uses an LLM to propose names from the class labels of top-activating images for fine-grained recognition. Experiments use ImageNet-1K, CUB-200, and Places365 with CLIP ViT-B/16 and ViT-L/14, comparing the proposed variants with a task-specific SAE baseline, DN-CBM, and linear probes. The paper reports classification accuracy, CLIP-based semantic-alignment scores, sparsity–accuracy and sparsity–reconstruction trade-offs, and qualitative concept examples. Its results suggest that label impurity often improves classification and semantic-alignment scores over the SAE baseline; the binary variant uses fewer active factors, while selected qualitative examples show visually coherent concepts.

Critical strengths

Strength 1:

The paper introduces label impurity, a clear supervised objective that encourages within-class consistency of factor activations while still allowing factors to be shared across classes (Eq. (3), p. 5). It also derives an unbiased degree-two U-statistic estimator that can be computed in linear time in the minibatch size (Theorem 1 and Appendix B, pp. 5 and 13--14). This is a principled and technically useful way to inject label information into sparse autoencoder-based concept discovery.

Strength 2:

The empirical study covers ImageNet-1K, CUB-200, and Places365 with both CLIP ViT-B/16 and ViT-L/14, and combines classification, semantic-alignment, sparsity, and qualitative cross-dataset analyses (Tables 3--4 and Figures 3--5, pp. 8--9). The results show that adding label impurity often improves over the task-specific SAE baseline, and the qualitative examples suggest that several learned factors are visually coherent.

Critical weaknesses

Weakness 1:

The evaluation protocol appears to use test accuracy for model or hyperparameter selection. The paper states that “all analyses including Sections 4.2.2 and 4.3 use the parameter settings that yield the highest test accuracy” (p. 7). At minimum, this makes the semantic and qualitative analyses in those sections dependent on test-set model selection; it also raises a concern about the selection protocol for the classification results in Table 3. Such selection can introduce optimistic bias and makes the reported evaluation non-independent. Appendix A states that a 99:1 training/validation split was already created, which could have been used for hyperparameter selection, making this test-set dependence avoidable.

Weakness 2:

The semantic-alignment evaluation is not fully independent of the supervision and representation used to construct the concepts. Factors are ranked by their importance for predicting class labels (p. 7); for general datasets, names are selected by maximizing cosine similarity to CLIP text embeddings (Eq. (6), p. 6); and the resulting concepts are evaluated using CLIP image--text similarity (p. 7). For CUB-200, names are instead generated from class labels of top-activating images (p. 6). Reusing class labels and CLIP text space in both concept selection/naming and evaluation may reward label or CLIP-space alignment rather than independently validated human interpretability. The paper provides neither human agreement scores nor a label-independent interpretability benchmark, so the strength of the semantic claims remains uncertain.

Weakness 3:

The quantitative evidence for semantic alignment is limited in scope and sampling. The paper evaluates this metric using a single sampled image per class (p. 7), while Table 4 reports results only for ImageNet and CUB-200, not Places365 (p. 8). No uncertainty across image sampling or training runs is reported. As a result, it is difficult to determine how stable the semantic-alignment gains are or whether they extend to the scene-recognition dataset; this weakens the breadth of the paper's empirical support for improved interpretability.

Questions for authors

Question 1:

Did the selection of the “parameter settings that yield the highest test accuracy” use test-set labels? If so, can the authors rerun model selection using only the training/validation split and report results on an untouched test set?

Question 2:

Which data split is used for factor ranking, concept naming, and image sampling in the semantic-alignment evaluation? How do the authors prevent label-driven factor selection and label-driven naming from making the evaluation circular?

Question 3:

Are Tables 3--4 averaged over multiple random seeds? If so, please report standard deviations or confidence intervals; if not, how sensitive are the reported improvements to SAE initialization and minibatch sampling?

Question 4:

Can the semantic-alignment evaluation be repeated over multiple sampled images per class, and can the authors clarify why Table 4 omits Places365? These results would help establish whether the reported gains are robust to image sampling and apply beyond the two evaluated datasets.

Reviewer_1BapOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper adds class supervision to sparse autoencoders (SAEs) for concept discovery in CLIP image embeddings. It proposes label impurity (LI), which pushes each factor's class-conditional activation rate towards 0/1 while sharing a factor across classes. Experiments on ImageNet, CUB-200 and Places365 report better linear-probe accuracy for SAE+LI and better semantic alignment for the binary model than a plain SAE and DN-CBM.

Critical strengths
  • Using class labels as weak supervision while allowing factors to be shared across classes is a reasonable middle ground between unsupervised SAEs and concept-supervision methods.
  • The objective is cheap and drops into any SAE.
Critical weaknesses
  1. The objective and the estimator are not new, and the paper does not position them. For binary codes, LI equals the expected within-class variance of each factor, E[Var(zj∣Y)]\mathbb{E}[\mathrm{Var}(z_j \mid Y)], which is the within-class scatter penalized in discriminative dictionary learning [1, 2] and center loss [3]. None of these is cited or compared.
    • With ReLU codes (SAE+LI), class means can exceed 1, where q(u)=u(1−u)q(u) = u(1-u) is negative, so LI rewards larger activations instead of class-consistent ones.
  2. The evaluation does not show that the factors are meaningful concepts rather than groups of classes. LI is minimized by any factor that depends only on the class, and the evidence points towards this partial collapse.
    • Reconstruction gets worse as we introduce LI and HTAF (Figure 3). Adding LI alone already costs reconstruction, and binarizing the factors costs much more, so the proposed model keeps little of the embedding beyond class information.
    • The factor names are generated from the classes on which a factor is activated (L306-309), so each name can be biased toward describing a group of classes by construction.
    • The comparisons are not controlled. SAE is never trained below about 1500 alive nodes in Figure 3.

[1] Meng Yang, Lei Zhang, Xiangchu Feng, and David Zhang. Fisher Discrimination Dictionary Learning for Sparse Representation. ICCV 2011.

[2] Zhuolin Jiang, Zhe Lin, and Larry S. Davis. Learning a Discriminative Dictionary for Sparse Coding via Label Consistent K-SVD. CVPR 2011.

[3] Yandong Wen, Kaipeng Zhang, Zhifeng Li, and Yu Qiao. A Discriminative Feature Learning Approach for Deep Face Recognition. ECCV 2016.

Questions for authors
  1. What is special about LI that makes unbiased estimation difficult?
  2. Can you compare against cross-entropy supervision on accuracy (Table 3) and semantic alignment (Table 4)?
  3. In Figure 3, is there a reason that no SAE configuration has fewer than about 1500 alive nodes? Please add SAE runs with a stronger sparsity penalty.
  4. Could the factors collapse to the class, so that the factor that best explains a cat image is that it is a cat?
  5. There are other ways to encourage class-wise activations, such as discriminative dictionary learning [1, 2] and center loss [3]. Please discuss them and compare with at least one.
  6. For SAE+LI with ReLU, how is LI kept from rewarding large activations? Is the decoder norm constrained?
  7. Minor: "Heavyside" should be "Heaviside" on L177 and L183, and "Baslines" (L315) should be "Baselines". The model is called "SAE+Heaviside+LI" in the text and "SAE+HTAF+LI" in the tables.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.