acceptodds
Withdrawn from ICLR 2027

Archetypes of a Concept: Sample-Traceable Decomposition of Concept Representations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Concept-direction methods summarize a concept with a single vector. This representation is compact, but it obscures variation within the concept and the training examples that support that variation. We introduce Archetypes of a Concept (AoC), a post-hoc method that decomposes concept-positive features into simplex-constrained archetypes. Each archetype is a convex combination of training features and therefore retains explicit sample-level provenance. A fixed concept probe then selects aligned archetypes. New queries are represented by simplex coefficients over the selected dictionary. We establish convex-hull traceability and a margin-based condition for preserving fixed-probe predictions. We also analyze coefficient continuity and selection stability under explicit regularity and score-gap conditions. Together, these properties support evidence inspection and coefficient-space analysis in a frozen feature space.

Primary area:
interpretability and explainable AI
Keywords:
concept-based interpretability, archetypal analysis, concept refinement, explainable AI, prototype retrieval, representation learning, vision-language models

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_GcTnOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper argues that having a concept direction summarized by a single vector is limiting. They introduce Archetypes of a Concept (AoC) as a post-hoc method that decomposes concept-positive features to create a new query. The procedure starts with training a linear probe, fitting sample weights and training-sample coefficients to positive training features, ranking archetypes and retaining from a selected dictionary, infer the coefficients over the dictionary from the query, and then compare queries over their coefficient vectors. The authors argue that this approach improves on prior work by having more utility and semantic meaning.

Critical strengths

Thorough description of AoC procedure with mathematical results to motivate the steps. Thorough description of prior work with explanations on how AoC differs from these methods.

Critical weaknesses
  • Paper is difficult to understand. There is no clear motivating impact, with introduction sparse on citations of why having this decomposition by training set is so important.
  • Paper relies on concrete training set (i.e. is not a theoretical contribution). However, there are no experimental results on real datasets on the results. There are also no benchmark results against other methods. While there are some results in the supplement, I do not believe the reader should have to shuffle through the supplement to find support for the primary claim of the text.
  • There is a discussion on AoC outputs and their semantic value without any example or explanation of a typical AoC output.

Without a clear motivation of the problem, results on data, or results on benchmarks, I can not recommend accepting this paper.

Questions for authors

There seems to be a lot of reliance on the training data for AoC. How do you expect AoC to act on distribution shifts? Is access to the full set required? Did you try sub sampling the training set and comparing results (i.e. robustness testing)?

Can you provide a running example on where AoC would be used, the outputs of AoC, and what impact that would have?

Why did you pick CRAFT as the only comparator in the Appendix compared to the other benchmarks (ACE, TCAV, etc) in your related work?

What set of models would this work over? CLIP models only?

Reviewer_BwbfOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Archetypes of a Concept (AoC) is a post-hoc representation method built on a frozen visual encoder and a binary concept probe. It fits simplex-constrained archetypal analysis to concept-positive features, so each archetype is a convex combination of training examples. It then ranks archetypes by alignment with the fixed probe and represents new queries by coefficients over the selected archetypes. The paper gives convex-hull provenance and conditional results for probe-prediction preservation and stability. Experiments on CUB and DTD examine fitted coefficient-space separability, sample evidence, retrieval, and perturbation behavior.

Critical strengths
  1. The method clearly separates an archetype’s training-sample weights from a query’s inferred coefficients. This distinction makes the claimed provenance precise: the weights show how an archetype was constructed, without presenting them as causal influence on a prediction. Combining this provenance with a fixed concept probe is a coherent inspection interface.
  2. The evaluation examines several properties rather than relying on one score, and the authors report unfavorable as well as favorable findings. The AoC–CRAFT comparison favors AoC on the two reported CUB settings but is mixed on DTD; the paper also acknowledges correlated non-target attributes and degradation under perturbations. These observations help define the method’s scope.
Critical weaknesses
  1. The principal quantitative comparison does not establish performance on unseen queries. The coefficient classifier is trained and evaluated on the same coefficient–label pairs (Section 5.1, Eq. 16), and configurations are selected by the largest fitted accuracy (Appendix K). Thus, the CUB gains of 6.6 and 4.7 percentage points over CRAFT in Table 2 measure in-sample separability after selection, not held-out prediction or generalization. This distinction is consequential because representing new queries is a central motivation of the paper. The main experimental section also contains no result figure or table.
  2. Mathematical sample traceability does not by itself establish meaningful concept-specific modes. Convex-hull provenance follows from classical simplex-constrained archetypal analysis, while the empirical margin certificate applies to only 265 of 6,702 reconstructed positive samples (Appendix I.1). The qualitative AoC–CRAFT panels select cases with large AoC differences; semantic matching relies on VLM judgments without human calibration; and the reported high prevalence of non-target attributes in support images indicates substantial entanglement. Moreover, the retrieval evaluation treats all other concept-positive images as relevant, which does not test whether coefficients recover distinct modes within the concept.
Questions for authors
  1. Can the authors evaluate the full pipeline on held-out images, with disjoint data for archetype and probe fitting, configuration selection, and final assessment? In particular, do the Table 2 AoC–CRAFT differences persist when kk, k′k', and classifier settings are selected only on validation data?
  2. What evidence shows that different archetypes correspond to stable, human-recognizable modes of the target concept rather than background or correlated attributes?
  3. How often do the conditions behind the prediction-preservation and stability results hold in the reported settings? The margin certificate covers only 3.95% of reconstructed positives; measurements of reconstruction margins, coefficient stability, and screening gaps would clarify the practical reach of the theory.
Reviewer_dwamOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper starts with a relevant limitation of Concept Activation Vectors (CAVs), which summarise a concept by a single direction in feature space without representing within-concept variation. The proposed method, AoC, decomposes single concept example representations approx. into a convex combination of positive concept examples. This is done by 1) solving an archetypal dictionary learning problem, where the archetypes are themselves convex combinations of positive concept examples and the reconstruction coefficients lie on the simplex, and 2) fitting a CAV with logistic regression and letting cosine similarity to CAV select top-k’ archetypes. They also propose how to decompose a new concept sample representation into archetype.

I see the central contributions as applying the idea of Archetypal Analysis(Cutler&Breiman, 1994) to positive examples of individual concepts and connecting the resulting decomposition to a discriminative concept direction via similarity screening. While this gives construction provenance for each archetype, the practical benefit of the screening mechanism and the usefulness of the query coefficients for characterising within-concept variation are insufficiently established.

Critical strengths

None

Critical weaknesses

Weakness 1 - Benefit of CAV screening is not established well and CAV could also be corrupted by background and spurious concept correlation artefacts:

One of the key contributions is 1) ‘probe-based archetype screening’ that should 2) ‘preserve the link to the original concept direction’ (Line 63-65).
Regarding 1), the motivation for CAV-based screening of the archetypes is that positive samples could contain background structure and spurious correlations. To filter out archetypes that represent these, they choose the top-k archetypes with maximal cosine similarity to the CAV. There is only an indirect evaluation of the benefit of CAV screening by comparing to random selection of archetypes at one fixed (k,k’) pair for one dataset/model combination by linear separability of positive and negative examples in concept coefficient space in L. 440/441. It would have been interesting, to compare to no screening and accuracy on the set of archetypes filtered out by CAV screening. Regarding 2): if I understand correctly, the CAV is trained on the same positive concept examples as the decomposition, so it is also subject to background structure and spurious correlations that could influence the direction of the CAV. Previous work by Pahde et al. 2022 suggests that alternative CAV extractors are more robust to artefacts in concept training data than logistic regression used in this work. As the alignment of the CAV with the true concept direction is important to AoC, an ablation of the concept extraction method would be important.

Weakness 2 - Experimental evaluation: Evaluations are either not motivated or not described at all. The majority of results is in the appendix and not referenced in the main text. There are only very few experimental results in prose and they are hard to decipher. Most importantly, as noted above, there is very limited demonstration practical benefit of the screening mechanism, and no demonstration of the usefulness of the query coefficients for characterising within-concept variation.

Based on these two major weaknesses, I would recommend to recent the paper.

[Pahde, Frederik et al. “Navigating Neural Space: Revisiting Concept Activation Vectors to Overcome Directional Divergence.” International Conference on Learning Representations (2022).]

Questions for authors

Clarification:

  • The connection between CAV and concept decomposition is only through the selection of k’ out of k archetypes by similarity to CAV, and k’<k. In practice, k,k’ are fitted individually for each concept. However, in L. 912-913, choices for (k,k’) pairs seem to include k==k’, which corresponds to no screening. Are these pairs included in the actual implementation of screening?
  • L2 norm regularisation of u (on the simplex) during inference of coefficients for a query feature promotes similarity to uniform coefficients u_i=1/k’. What is the motivation for this regularisation, why have the authors omitted it in their experiments (as stated in Table 5)? A lot of previous work on concept decomposition (not for data attribution decomposition though, as in this work) is on Sparse Autoencoders which regularise for sparsity of concept coefficients.

Suggestions:

  • Sec 3.2: The presentation of their method for concept extraction, which is just standard logistic regression on, is unnecessarily elaborate and in this way slightly confusing. In particular, the introduction of the “effective separation vector” seems to introduce standard re-parametrization as a novel methodological contribution. It would be better to call extraction method by its name and motivate the choice, ideally in comparison to previous approaches, e.g. Pahde et al. 2025 compares an array of different extraction methods for CAVs.
  • The theoretical analysis consists mainly of direct consequences of the simplex constraints and standard optimisation sensitivity arguments with limited novelty and relevance to AoC. Instead of using a considerable amount of space in the main paper for these, I would suggests to instead 1) motivate the methodological choices for CAV extraction and concept decomposition better and 2) show experimental results in the main test.
  • AoC’s connection between concept directions and concept training examples would benefit from comparison with SVM-based CAVs. A linear SVM admits an exact expansion of its concept direction over support vectors, providing a direct link between the probe and its training examples.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.