- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents a simple method for measuring the geometry of a concept in a given model's representation space. It targets both VLMs and LLMs. For a given model (say a VLM), a dataset and a concept c (for VLMS, concepts come from Visual Genome), they first select positive (image contains c) and negative images (image does not contain c). Then, they compute simple statistics of the model representations for the positive set. This is called "locating the concept geometry". THis geometry is defined by the normalized mean of the unit-normalized samples; the largest cosine distance from any sample to that axis; and, the smallest and largest sample norms. They can also track these statistics during training. They also have a disentanglement score which measures a kind of coverage between positive space and negative space. They report results on various CLIP, Llava and Qwen models.
Critical strengths
The paper tackles an important problem: understanding how concepts are encoded in the interal representations of models.
Proposed method is simple and experiments have breadth (CLIP, Qwen, Llava; visual concepts, text concepts, etc.)
Critical weaknesses
The paper does not compare any of its results with any existing method. It neither has a sensible baseline. Therefore, it is not possible to assess whether this paper makes any progress over existing methods.
The paper draws inspiration from "diverse geometric views of concept representations (Engels et al., 2025; Modell et al., 2025; Sorscher et al., 2022), ..." However, it does not compare any results with them. Each of these references is a potential competitor. Especially Sorscher et al., since they fit an ellipsoid per concept.
Suppose there is no existing method to compare against (which is not the case), then there should be sensible baseline. For example, a random-concept control, where capsules are fit to random samples of the same size drawn from the whole dataset, and compute disentanglement against random negatives.
The comparison with cones, balls and boxes in Section 3.3 only ablates the authors’ own construction; it is not a comparison with existing methods or a control.
About negative images: the paper only says negatives are “examples that do not belong to concept c” . It doesn’t say how they are sampled, but this is crucial detail missing. For example, whether you choose hard negatives would have a drastic effect on disentanglement.
Questions for authors
How were the negatives sampled?
What does the random-concept control give?
How do capsules compare with an ellipsoid or a linear probe at the same coverage?
I should note incorporating the answers of these questions to the paper is more than a major revision.