- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses the ambiguity of interpreting each latent of an SAE trained on vision models using top-activating examples, where multiple visual attributes may co-occur, making it difficult to identify the precise semantics. To address this problem, the authors propose RECSAEI, which intervenes on SAE latent directions and reconstructs the modified representations into images to reveal their semantic meanings. The paper also introduces a method for intervening on [CLS] tokens by attributing them to patch-level SAE activations. The authors quantitatively shows better in interpreting concepts.
Critical strengths
- The paper addresses a meaningful limitation of interpreting SAE latents using only top activating examples
- The authors attempt to quantitatively evaluate the additional information provided by intervention reconstructions
Critical weaknesses
- The practical benefit of intervention reconstruction is not sufficiently established.
- Although the ambiguity of top activating examples is a valid concern, it is difficult to grasp how severe this problem is across the SAE dictionary. Without this information, the reviewer cannot agree that the problem is significant
- The improvements from intervention reconstruction in Table 1 appear relatively marginal, mostly around 1–2 percentage points compared to top-bottom, raising questions about their practical significance. Moreover, it is unclear whether these gains come from intervention-specific information or simply from providing additional inputs. Comparing against additional top/bottom examples under a matched input budget would help clarify its contribution.
- The proposed benchmark measures whether an LLM can identify which image has higher SAE activation, but it is unclear whether improved activation prediction accuracy necessarily reflects a more accurate interpretation of the visual concepts represented by the latent. Is the benchmark properly designed so that humans can also achieve reasonable scores? Even a small-scale human evaluation of whether intervention reconstructions help distinguish the visual concepts represented by ambiguous features would provide more direct evidence.
- The reported improvements are relatively small, but the paper does not establish whether they are statistically significant or robust to different samples of SAE features and image pairs. Confidence intervals and evaluations across different random samples would help establish the reliability of the results.
- The [CLS] intervention experiments do not sufficiently support the paper's main interpretability objective.
- While the experiments show improved activation control and provide qualitative analyses of contributing patch-level features, it remains unclear whether these results lead to more accurate semantic interpretations of [CLS] features. Since the paper's primary objective is clear latent interpretation, directly evaluating whether intervention reconstructions help identify the visual attributes represented by [CLS] features would better support its central claim. Also the visual differences in examples are too small, and it is unclear whether they really matter to human interpreters.
Questions for authors
- How widespread is the ambiguity of top activating examples across SAE latents? Can the authors estimate how frequently this issue arises beyond the selected qualitative examples?
- Could additional top/bottom examples or carefully selected contrastive examples (exclusively activating samples) resolve similar ambiguities? Does intervention reconstruction still provide a benefit when compared against these alternatives under a matched visual input budget?
- How well does the proposed activation prediction benchmark reflect semantic interpretability? Even a small-scale human evaluation comparing latent interpretations with and without intervention reconstructions would help validate this connection.
- Are the improvements in Table 1 statistically significant and robust to different random samples of SAE features and image pairs?
- How does the proposed sparse steering method improve the semantic interpretation of SAE latents?
- How do the authors distinguish meaningful visual changes associated with the target SAE latent from artifacts introduced by the RAEv2 decoder?
5 comments
Author Response (2/2)
W1.4/Q4. Statistical significance
We assess statistical uncertainty using dimension-level bootstrapping with 20,000 replicates. We resample SAE dimensions while retaining their associated image pairs, which were selected to satisfy the benchmark constraints and maximize the target feature's activation contrast. The confidence intervals are reported below.
| Dataset | Backbone | Top | Top + Intervention | Top + Bottom | Top + Bottom + Intervention | Gain |
|---|---|---|---|---|---|---|
| NABirds | DINOv3 | 66.3 [64.7, 67.9] | 68.2 [66.6, 69.7] | 73.0 [71.5, 74.5] | 74.8 [73.4, 76.3] | +1.80 [+0.66, +2.92] |
| NABirds | SigLIP 2 | 61.0 [59.3, 62.7] | 63.8 [62.2, 65.4] | 67.9 [66.4, 69.4] | 69.2 [67.7, 70.6] | +1.26 [+0.14, +2.40] |
| CUB | DINOv3 | 64.3 [62.7, 65.9] | 65.4 [63.8, 67.0] | 70.3 [68.9, 71.8] | 71.4 [70.0, 72.8] | +1.08 [-0.06, +2.22] |
| CUB | SigLIP 2 | 63.1 [61.5, 64.7] | 64.8 [63.3, 66.3] | 69.7 [68.3, 71.2] | 71.1 [69.6, 72.5] | +1.30 [+0.22, +2.40] |
| Cars | DINOv3 | 67.4 [65.9, 68.8] | 68.4 [67.0, 69.8] | 73.2 [71.8, 74.5] | 74.3 [72.9, 75.6] | +1.08 [-0.06, +2.23] |
| Cars | SigLIP 2 | 65.5 [64.0, 67.0] | 66.0 [64.5, 67.5] | 69.4 [67.9, 70.8] | 71.0 [69.6, 72.4] | +1.60 [+0.50, +2.69] |
| ImageNet | DINOv3 | 74.7 [73.3, 76.1] | 71.4 [69.9, 72.8] | 76.9 [75.6, 78.2] | 76.2 [74.9, 77.5] | -0.70 [-1.74, +0.36] |
| ImageNet | SigLIP 2 | 71.8 [70.4, 73.3] | 72.3 [70.8, 73.7] | 73.5 [72.1, 74.9] | 73.9 [72.5, 75.3] | +0.34 [-0.64, +1.32] |
Adding interventions to Top + Bottom improves interpretation accuracy across all six settings on the fine-grained NABirds, CUB, and Cars datasets, with an average gain of 1.4. Four improvements have 95% confidence intervals excluding zero, while two additional improvements (CUB and Cars with DINOv3) have lower bounds close to zero (−0.06 percentage points). These results provide statistical support for the benefits of intervention explanations, particularly on fine-grained datasets.
We have added the statistical significance to Appendix E.2.
W2. [CLS] explanation
W2.1/Q5. Quantitative experiments for [CLS] explanation
We thank the reviewer for this suggestion. To directly evaluate whether intervention reconstructions improve the interpretation of [CLS] SAE features, we constructed a similar paired benchmark on NABirds containing 1,000 blind image pairs across 200 [CLS] features.
| Explanation Method | Accuracy (%) |
|---|---|
| Top | 57.5 |
| Top + Interventions | 58.5 |
| Top + Bottom | 61.1 |
| Top + Bottom + Interventions | 62.1 |
Using intervention reconstructions improves both baselines by 1.0%. Notably, on 190 pairs where top-example and top–bottom explanations yield different predictions, interventions improve accuracy from 40.5% to 51.1%. These results suggest that intervention reconstructions provide complementary visual evidence for interpreting [CLS] features, particularly when top- and bottom-examples are ambiguous.
W2.2. Small visual differences in [CLS] explanation
The subtle visual differences are partly attributable to the fine-grained nature of the evaluated domains, where meaningful attributes often involve small variations in morphology, coloration, or texture. Moreover, the magnitude and semantic coherence of the reconstructed changes depend on the underlying SAE features. Thus, subtle or ambiguous changes may also reflect limitations of the learned SAE representations, instead of the intervention method itself. Our method aims to reveal the visual information encoded by existing SAE features, including fine-grained attributes whose effects may be difficult to perceive. Using the resulting visual changes as an approach to evaluate the quality of SAE representations can also be an important future direction.
We have included the experimental results and discussion in Table 3 in Section 4.3.
Q6. Meaningful visual changes vs. reconstruction artifacts
We thank the reviewer for this question. Our intervention analysis directly compares two reconstructed images generated by decreasing and increasing the target SAE feature activation. Both images are produced using the same RAEv2 decoder, with interventions targeting localized visual changes while preserving the remaining image content. This provides a direct contrast that helps distinguish feature-associated changes from shared reconstruction artifacts. Although this comparison cannot eliminate decoding artifacts, visual changes across intervention strengths provide direct evidence to associate with the target SAE feature.
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts, and I believe the paper has improved. However, some concerns remain.
First, while the additional experiments provide stronger evidence for the benefits of intervention reconstructions, the improvements remain relatively small, typically around 1–2 percentage points. In the human study, adding interventions slightly increases correct responses, but also increases incorrect responses. This raises questions about whether the method meaningfully improves interpretation reliability.
Second, in W1.1, the authors state that "the ambiguity of top-activating examples is particularly relevant to fine-grained domains." However, it remains unclear whether the underlying foundation models and SAEs reliably capture meaningful fine-grained attributes in the first place. As the authors acknowledge in W2.2, subtle visual changes may reflect limitations of the learned SAE representations. If the foundation models or SAEs fail to learn these fine-grained attributes, it is unclear how much practical value intervention reconstructions can provide in interpreting them.
Author Response (1/2)
Dear Reviewer aEZ9,
We sincerely thank the reviewer for the constructive feedback and for recognizing the significance and the quantitative experiments in our paper. Our responses to the questions are listed below.
W1. Practical benefit of intervention reconstruction
W1.1/Q1. Ambiguity of top-activating examples across SAE latents
We thank the reviewer for this question. The ambiguity of top-activating examples is particularly relevant to fine-grained domains. We find that a typical SAE feature on NABirds (DINOv3) is activated by 108 of 404 categories (27%), compared with 58 of 1,000 categories (6%) on ImageNet. This suggests SAE activations are distributed across many categories, and the semantics are more difficult to infer based on the category label. Furthermore, the median SAE feature activates on only 2.7% of image patches for NABirds, indicating that the features might be very localized. The difficulty is also reflected in our quantitative benchmark: using top-activating examples alone achieves 66.3%, 64.3%, and 67.4% interpretation accuracy on NABirds, CUB, and Cars, respectively, compared with 74.7% on ImageNet.
While these results characterize the difficulty of interpreting fine-grained SAE features, directly quantifying the prevalence of ambiguity would require reliable independent interpretations of individual features. Our intervention reconstruction provides complementary visual evidence by modifying the target feature, helping distinguish its effects from co-occurring attributes. Our experiments show consistently improved interpretation accuracy on fine-grained datasets.
W1.2/Q2. Practical significance of intervention reconstructions
We thank the reviewer for this suggestion and present the results below.
Explanation budget
We first increase the number of image–overlay pairs to 10 for top, so the total number of images matches the 20 images for top + overlay + intervention. Similarly, we increase the number of explanation examples to 9 for top + bottom, resulting in 27 images in total vs. 25 images with intervention reconstructions. We report the results in the table below.
Increasing the number of examples has a marginal influence on the prediction accuracy. These results suggest that the benefit of intervention explanations cannot be explained simply by providing more examples.
We have added the explanation budget ablation study to Appendix D.2.
Contrastive examples
We further investigate whether selecting more discriminative examples for explanation can resolve ambiguities. We use the same approach of constructing the blind benchmark to select top–bottom pairs, but from the training set. The resulting top and bottom examples have high overall similarity, but differ in the target dimension. However, the accuracy on NABirds is 71.3%, below the 73.0% achieved with standard top- and bottom-activating examples. This suggests that selecting examples with stronger activation contrasts does not necessarily produce more informative explanations. The main reason is that the image pairs that are overall similar often cannot create the sharpest contrasts in the target dimension. Comparatively, interventions can modify individual dimensions, and thereby create more targeted contrasts to help interpret sparse features.
We have added the contrastive-explanation experiment results to Appendix D.3.
W1.3/Q3. Human study
We thank the reviewer for this suggestion and have run a human study with the same protocol as the LLM judge. Participants saw the explanation for a latent and a held-out image pair, and chose which image activates the latent more strongly; they could also abstain with “Hard to tell”. We collected 360 responses over 259 latents.
When participants chose an image, they were correct 81–86% of the time in every condition, far above the 50% chance level. The results indicate that humans can produce meaningful evaluations for this benchmark. Their errors also coincide with those of the LLM judge: the judge is correct on 71–82% of the pairs humans answered correctly, but only 38–65% of the pairs they answered incorrectly.
The number of correct answers increases from top examples (71) to top + bottom examples (82) to intervention reconstructions (84), the same ordering the LLM judge shows on the full benchmark (66.3% → 73.0% → 74.8%). We will extend the study with more participants and test dimensions.
We have added the human study results to Appendix E.1.