- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces PACO-SAE, an object-level benchmark for evaluating whether vision SAE features are truly grounded in the visual concepts they represent, rather than merely predicting concept presence. It further analyzes feature splitting across object instances and proposes Pairwise Visual Separation (PVS) to distinguish meaningful visual specialization from residual splitting.
Critical strengths
- The paper is motivated by a clear and important limitation of existing vision SAE evaluation: strong concept prediction does not necessarily imply correct spatial grounding.
- The proposed benchmark and evaluation protocol are well designed, with exhaustive object annotations, held-out feature selection/testing, and object-level grounding metrics that make the conclusions convincing.
- The experiments are comprehensive and insightful, especially the analysis showing extensive feature splitting and the human-supported PVS metric for examining whether such splits correspond to meaningful visual specialization.
Critical weaknesses
- The evaluation is limited to a single visual backbone and specific layers. Experiments were conducted solely on the 11th block of DINOv3 ViT-B/16. Evaluating additional backbones (such as CLIP-based models) and various layers would significantly strengthen the generalizability of the conclusions.
- The feature splitting analysis based on an Oracle relies on ground-truth masks, potentially overstating the practical significance of the splitting phenomenon. The metric improved substantially by utilizing ground-truth segmentation masks to select the optimal feature for each test object. While this demonstrates that distinct features can localize different instances, it does not show whether such instance-specific feature selection is achievable without Oracle supervision. Therefore, the reported 39%–43% performance gain should be interpreted with caution regarding its real-world implications.
- PVS measures the degree of specialization within the backbone's representation space rather than directly within the SAE feature space. PVS is calculated based on the distances between object representations derived from backbone features prior to the SAE stage. Thus, a high PVS value merely indicates that the backbone has visually distinguished between two sets of objects; it does not necessarily prove that the SAE has learned semantically meaningful, specialized features. The quality of the paper would be enhanced by supplementary analysis demonstrating that the observed specialization is specifically induced or captured by the SAE, rather than simply inherited from the backbone's representations.
Questions for authors
See the weakness above.
2 comments
Response to Reviewer R6Cv (2/2)
W3. PVS may reflect backbone geometry rather than specialization captured by SAE features.
Response: Thank you for this insightful comment on the relationship between the SAE backbone and the PVS representation, and for motivating a comparison using SAE feature parameters.
1. SAE features determine the object groups; PVS evaluates their visual differences. Under our oracle protocol, objects of each concept are grouped by the SAE feature with the highest grounding F1 against their masks. PVS evaluates the groups' visual separation relative to within-group variation. Although pretrained representations already contain visual structure, the groups are determined by the SAE's grounding behavior. Different SAE features can best ground visually similar objects and yield low PVS, illustrating residual splitting. We therefore use PVS to assess the degree of visual specialization captured by SAE features.
2. PVS remains informative when its evaluation space differs from the SAE backbone’s representation space. We make this distinction explicit in revised §3.4. We conducted an additional human evaluation of CLIP-trained BatchTopK at , with DINOv3 fixed as the PVS extractor (revised Figure 6). The retained samples comprise 196 CLIP pairs and 195 DINOv3 pairs:
| SAE backbone | Q1 (lowest PVS) | Q2 | Q3 | Q4 | Q5 (highest PVS) |
|---|---|---|---|---|---|
| CLIP | 0.08 [0.03, 0.21] | 0.33 [0.20, 0.48] | 0.30 [0.18, 0.45] | 0.31 [0.19, 0.46] | 0.56 [0.41, 0.71] |
| DINOv3 | 0.10 [0.04, 0.24] | 0.28 [0.16, 0.43] | 0.33 [0.21, 0.49] | 0.46 [0.32, 0.61] | 0.63 [0.47, 0.77] |
Majority-Yes specialization rates; reported intervals in brackets. DINOv3 is the PVS extractor for both rows.
CLIP shows higher specialization in Q5 than Q1, with similar rates in Q2–Q4. Thus, the human-supported association persists when the SAE backbone and PVS extractor differ.
3. Comparison with SAE decoder directions. Following your suggestion, we compared PVS with decoder cosine distance, , between SAE decoder directions in the input representation space. On the same 195 DINOv3 pairs, Pearson correlation with the human Yes fraction is 0.363 for PVS versus 0.175 for decoder cosine distance (revised Table 3). This higher observed association supports using the rich visual information in the objects each feature grounds to evaluate specialization beyond decoder similarity alone.
Response to Reviewer R6Cv (1/2)
Thank you for recognizing the importance of distinguishing concept prediction from spatial grounding, the benchmark's annotation and evaluation protocol, and the feature-splitting analysis. We address your three concerns below.
We extended the evaluation to CLIP ViT-B/16 across all four SAE families and k∈{32,64,128,256}, and to DINOv3 layers 9 and 10 with BatchTopK-128, using the same selection/test protocol.
The grounding advantage generalizes to every additional setting. CLIP also reproduces substantial splitting: object-wise oracle F1 exceeds fixed-feature F1 by 0.187–0.244 across its 16 settings. (Revised Tables 1–2, §4.1–4.2, Figures A4, A6, and A8.)
Thank you for emphasizing the practical interpretation of the oracle analysis. Our analysis asks whether one feature selected for a concept grounds its different objects well, or whether different objects are better grounded by different features. Ground-truth masks let us measure this difference. The reported 39–43% relative F1 improvement quantifies how much a single concept-selected feature misses relative to other existing features; it is a diagnostic upper bound, not a demonstrated test-time improvement. We agree that this distinction matters for real-world implications and have emphasized it in the revised §4.2.
Inspired by your emphasis on real-world implications, we conducted a proof of concept using Grounding DINO [1] and SAM [2] to recover part of the oracle gain without ground-truth test masks for feature selection. We randomly sampled 1,260 test objects—20 from each of the 63 concepts—and evaluated DINOv3 ViT-B/16 BatchTopK at k=128. For each existing benchmark crop, we provide the known concept name to Grounding DINO and use the predicted box nearest the crop center to prompt SAM. From the same 32 candidates fixed on the selection set, we select the feature whose activation mask has the highest F1 against the resulting pseudo mask. Feature choices are fixed before evaluation against ground-truth test masks. If no box is detected or the projected pseudo mask contains no foreground patch, we retain the concept-selected feature, jground.
For each evaluation subset S, we define the oracle-gain recovery as
Recovery(S)=Goracle(S)−Gfixed(S)Gpseudo(S)−Gfixed(S)×100
Each G is grounding F1 averaged over objects within each concept and then equally over the 63 concepts. All three terms use the same objects in S.
The usable-mask subset includes all objects for which pseudo-mask matching was possible, including inaccurate masks. Recovery is calculated from unrounded F1 scores. Thus, pseudo masks recover 28.33% of the oracle gain when usable masks are available, and 21.92% overall, including the 233 cases without a usable mask. Although this recovers only part of the gap, it shows that automatically generated masks can support per-object feature selection without ground-truth test masks. We believe improving this approach is an interesting direction for future work, which we highlight in the revised §5.
[1] Liu et al. Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection. ECCV 2024.
[2] Kirillov et al. Segment Anything. ICCV 2023.