- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates how DINOv2 uses visual concepts that are unique to individual classes or shared across multiple classes. Its main methodological contribution is Multi-level Clustering with Peeling (MCP), which repeatedly clusters classes and removes shared concepts to build a multilevel structure of class relationships. The authors evaluate the resulting clusters using semantic similarity measures, clustering baselines, and concept-removal experiments. They find that removing class-unique and shared concepts together often causes a larger accuracy drop than the sum of removing either group alone. Based on these results, the authors make their main scientific claim: DINOv2 uses class-specific and shared evidence simultaneously through a process they call "parallel abduction."
Critical strengths
The authors systematically measure the effects of removing class-unique and shared concepts from DINOv2 representations. The experiments reveal substantial differences across classes in their reliance on these two concept groups. The joint removal experiments also show that both groups can contribute important information to classification. These findings provide useful empirical insights into how visual concepts support recognition, even though they do not establish the proposed parallel abduction mechanism.
Critical weaknesses
- The central claim of parallel abduction is not supported by the experiments.
The primary evidence for parallel abduction is the super-additive accuracy loss that occurs when both the class unique and cluster unique concepts are removed. There's actually a far simpler explanation for this phenomenon in terms of the classification margin of a standard linear classifier. The two concept groups additively contribute to the logits. You can remove one of the groups and still have a positive classification margin. Remove both and the margin will go negative and you get a misclassification. For instance, if the original margin is 3 and each concept group contributes 2 to that margin then removing either one of the groups will leave you with a margin of 1 (still positive). Remove both and you end up with a margin of -1 (negative). So you can quite easily get that super-additive degradation in accuracy without any abductive mechanism at all. The experiments do show that the two concept groups are providing complementary information for classification but they don't demonstrate that the model is doing parallel abduction any more than they show that it's doing ordinary discriminative classification. That really undercuts one of the paper's key claims.
- The novelty and necessity of MCP are insufficiently established.
MCP is a combination of existing sparse subspace clustering with a set of heuristic rules for picking the clustering resolutions and for iteratively removing concepts. There's no principled rationale given for the choice of these rules and no convincing evidence that the resulting structure is better than simpler alternatives such as graph based clustering or simply filtering concepts by their firing set size. More fundamentally, the key class level ablations in Section 4.2 don't require MCP at all. Under the peeling rule a class stays active until all of its linked concepts have been peeled. As MCP will eventually peel every shared concept, the full set of cluster unique concepts linked to a class is precisely the original set of shared concepts in the biadjacency matrix . The main class level ablation results can therefore be derived directly from without any need for clustering or peeling. This really does cast serious doubt on what MCP is actually contributing to the paper's main results and whether the added complexity of MCP is worth it.
Questions for authors
- Parallel abduction
The super-additive accuracy loss seems to have a much simpler explanation. In a linear classifier you could remove either of the concept groups and still have a positive classification margin. You could remove both and end up with a negative margin. That doesn't require any abductive reasoning at all. What evidence do the authors have to support their interpretation of this as parallel abduction rather than this much simpler explanation?
- MCP's role in Section 4.2
Since MCP ultimately strips away all the shared concepts, isn't the full set of cluster-unique concepts associated with a class exactly the same as its original set of shared concepts in ? If that's the case then the class level ablations in Section 4.2 could be carried out without MCP at all. What exactly is MCP doing in those experiments?
- Iterative peeling
Why is iterative peeling necessary? A much simpler approach would be to just filter the concepts on the basis of their firing set sizes and then cluster the classes at various resolutions. Does MCP actually provide any advantage over that approach?