acceptodds
Withdrawn from ICLR 2027

Dual Convex Composition for CLIP Interpretability

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Concept decompositions of CLIP need both inspectable provenance and a clear scale for their activations. We present DCC-CLIP, which constructs each concept as a convex mixture of in-domain sample embeddings and reconstructs each sample as a convex mixture of concepts. A cosine objective fits the directional representation while the two weight systems expose dictionary provenance and sample composition. We establish hull containment, sparse representability, and fixed-weight perturbation bounds, distinguishing these properties from semantic interpretability and optimizer stability. Experiments compare adapted NMF/CRAFT and PCA on image classification and image–text alignment. DCC-CLIP obtains reconstruction cosine similarities from 0.947 to 0.993 across the reported dataset–backbone pairs. Semantic consistency, discriminative retention, and insertion/deletion evaluations examine complementary aspects of the resulting explanations. Code is provided in the supplementary material.

Primary area:
interpretability and explainable AI
Keywords:
CLIP, interpretability, concept decomposition, convex geometry, multimodal embeddings, vision-language models, explainable AI

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RrD8OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a double factorization to decompose image features into concepts, and decompose concepts into training image weights. This leads to a double attention mechanism, one with the concepts attending to the training images producing refined concepts, and one with the training images attending to the refined concepts. The objective is to reconstruct the training images by maximizing the cosine similarity to follow CLIP's direction. Specifically, given the features of CLIP image encoder of n images Z (shape nxd) the authors learn a C matrix (with softmax constraint) of shape (n x k) where k are the number of concepts. C shows which training samples construct each concept. By performing W = C^T.Z the authors attend over the training images, and produce a refined concept dictionary of shape (k x d). The authors also learn another matrix B of shape (n x k) also softmax constrained which shows how refined dictionary reconstructs each individual embedding, and by performing B.W they get the reconstructed Z features. The objective is to minimize the cosine similarity between the reconstruction and original training image features.

Critical strengths
  • The decomposition links learned concepts to their contributing training samples, which I think is novel and can aid the data attribution field. Different for other concept works, the method offers a mechanism for tracing learned concepts back to their contributing samples
  • The paper provides theoretical analysis
Critical weaknesses
  • As this is an incremental work on feature factorization, the authors should compare with other methods in the literature. Does their method provide an advantage over other methods? I do not see where the method shines. I dont see any evidence that the proposed method provides better interpretability. The authors compared only with CRAFT, but this is not a CLIP-based approach, and it is unclear how the authors adapted it to CLIP. Some existing works on interpreting CLIP with concepts are [R1 - R5]. These works are not even discussed in the related work. These works are even better because one can identify via text what each concept represents. On the other hand, the authors method needs manual inspection to get what this concept means.
  • In my opinion, the approach could be viewed as two single-layer transformers cross-attending to each other. The first transformers with queries as learable concepts and keys-values as image embeddings, producing a refined concept dictionary. The second transformer has queries as the image embeddings with keys-values coming from the refined concept dictionary from the first transformer. The only difference is that only the weight matrices for attention are learned. So essentially this is like adding some learnable layers on top of CLIP, like an adaptor, to adapt CLIP to other applications (here interpretability). This also means the authors should compare with CLIP adaptations for interpretability. Some of the works are [R6, R7, and R8]. For example, R6 trains a concept-based surrogate to reproduce CLIP's representation similarities and provide concept explanations and R7 learns a projection from CLIP embeddings into a human-interpretable concept space using alignment and reconstruction losses. These works highly overlap with the proposed method.
  • The fidelity and faithfulness of the reconstructed features is not proven with linear classifier training. Essentially you can change the whole distribution of CLIP, train a classifier on the new distribution and still get good results. I was expecting to see the zero-shot accuracy between the original CLIP and the reconstructed CLIP features. This is missing. This also means the insertion/deletion experiments cannot be trusted because they reply on the newly-trained classifier to extract confidence values, not the original zero-shot classifier (which is the text encoder of CLIP).
  • How does the method scale to large datasets? The authors only use a subset of ImageNet (100 images) but this is far from realistic, and the computation will increase for every sample as the method needs to learn a new matrix for each test sample. This is important are we are currently in the large-data regime.
  • Im not sure why gradients are used to identify the most important concepts. This disconnects the method from the way the authors select concepts. In my opnion the authors should evaluate by selecting the most important concepts from their learned matrix B, which identifies how strongly each concept participates in reconstructing a particular image. It measures concept activation. This is the correct way to evaluate this and select concepts. However, I do understand that this could lead to problems, because background concepts of the image (unrelated to a class) might not lead to a high class confidence, but still has a high reconstruction. Therefore, the authors can adapt the insertion/deletion metric but measure the reconsutrction of the features rather than the confidence metric, as concepts according to B are inserted/deleted. This correctly evaluates B. B-based ranking evaluates the importance of concepts to the learned reconstruction and should be reported. Essentially, another version of E1 where you gradually insert/delete concepts according to B, and measure this against the original CLIP embedding.
  • Interpretability is only useful if we can actually do something with it. Can you for example improve CLIP zero shot accuracy with the extracted concepts? In the recent few years tons of papers are our about analyzing and understanding models, but no one is actually doing anything with those findings. They are just archived and forgotten. The real value comes when you have a practical application of your approach such as improving classification, segmentation...etc

[R1] Interpreting and Analysing CLIP’s Zero-Shot Image Classification via Mutual Knowledge [R2] Interpreting CLIP with Sparse Linear Concept Embeddings [R3] Matryoshka Sparse Autoencoders [R4] Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discovery [R5] Interpreting CLIP's Image Representation via Text-Based Decomposition [R6] EXPLAIN-R: Advancing Interpretability of CLIP Representations with Concept Surrogate Model [R7] Explaining CLIP Zero-shot Predictions Through Concepts [R8] Interpreting CLIP with Hierarchical Sparse Autoencoders

Questions for authors

I think the paper is interesting but it currently has several problems: 1) no comparisons with similar works and evidence that the proposed method has better interpretability, 2) evaluation is wrong, as the authors train their own linear classifier rather than using the original CLIP zero-shot classifier (which is the CLIP text encoder), and 3) some other evaluations missing. Additionally it is unclear how the method can scale to large datasets. I would also like to see a practical application of the method rather than just interpretability.

Reviewer_tVzDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes to explain CLIP embeddings by first building each concept from a convex combination of training examples and secondly reconstructing each image as a mixture of concepts. It conducts experiments on CUB-200, ImageNet 100-class subset, Flickr8k and CC, using CLIP RN50 and ViT-B/16, compared with NMF and PCA.

Critical strengths
  • The theory for designing the method is clear enough.
  • It provides several comprehensive evaluations, including semantic consistency, reconstruction, and concept insertion/deletion.
Critical weaknesses
  • The dual convex-mixture structure is classical archetypal analysis, and the contribution is its adaptation to CLIP with a cosine objective and provenance interface. Therefore, the methodological novelty is limited.
  • I can understand the concepts with CUB-200 and ImageNet, where the training image embedding mainly points to a class, but when using Flickr8k and CC, one image may be complicated with many objects. Then what concept does one image in these two datasets represent?
  • The explanation is about the image embedding space. Since CLIP is a VLM, how are the explanation results on the text side for the text embedding?
  • The compared NMF and PCA are classical but not recent methods.
  • The content of Sec. 5.4 is empty.
Questions for authors

see the weaknesses

Reviewer_j5PuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors study the problem of interpretation of CLIP features. They propose DCC-CLIP, which constructs a dictionary of concepts based on the clusters of the features, and also decomposes each feature into a collection of concepts. Then they provide theoretical analyses on the geometric properties of the DCC-CLIP. Finally, they conduct experiments on different datasets to demonstrate the effectiveness of the proposed approach under different metrics.

Critical strengths
  • The paper is well motivated, and the problem is important.
  • The idea is simple and interesting.
Critical weaknesses
  • The paper is poorly written:
    • Some texts feel like an agent reporting to its user:
      • Line 223, "The supplied loader requests train/test caches, but its dataset module is absent, so exact sample identities, subset selection and split construction cannot be verified from this release."
      • Line 244, "A matched-input ablation remains necessary for that attribution."
      • Line 245, "The supplied metric-generation scripts are also incomplete, limiting verification of historical aggregation and insertion/deletion endpoints."
    • It seems Sections 3 and 5.4 are incomplete
  • The paper provides little detail about the proposed approach, except the objective function (4)
Questions for authors
  • Questions
    • How is DCC-CLIP implemented (pseudocode suffices)?
    • Why can we represent a concept as a convex combination of data features? And why can we represent a data feature as a convex combination of concepts?
    • How does DCC-CLIP promote the Interpretability?
  • Other issues
    • The references in Section 5 should be included in the parentheses. For example, 'CUB-200 Wah et al. (2011)' -> 'CUB-200 (Wah et al., 2011)'.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.