acceptodds
Under review as a conference paper at ICLR 2027

From Activation to Intervention: Interpreting Sparse Visual Features through Reconstruction

Abstract

Sparse autoencoders (SAEs) decompose visual representations into individual features, offering a promising basis for explaining model behavior. Interpreting these features typically relies on images or pixel regions that most strongly activate them. However, this evidence becomes ambiguous when multiple features with different semantics are frequently activated in the same locations, such as texture and color of the same patterns. We propose Reconstructive SAE Intervention (RecSAEI), a framework that shifts feature explanation from passive observation to active visual reconstruction. By intervening on representations along SAE decoder directions and reconstructing the visual output, we provide direct evidence of a feature's semantic meaning. We demonstrate that our method successfully disambiguates feature meanings across multiple datasets and vision encoders. Furthermore, our framework grounds [CLS] SAE interventions in a sparse set of patch SAE decoder directions. Beyond stable target response, this attribution also exposes the compositional relationship between elementary visual patterns and high-level semantics. Our results advocate for intervention reconstruction as an essential complementary signal for explaining neural networks.

Primary area:
interpretability and explainable AI
Keywords:
Interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aEZ9OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the ambiguity of interpreting each latent of an SAE trained on vision models using top-activating examples, where multiple visual attributes may co-occur, making it difficult to identify the precise semantics. To address this problem, the authors propose RECSAEI, which intervenes on SAE latent directions and reconstructs the modified representations into images to reveal their semantic meanings. The paper also introduces a method for intervening on [CLS] tokens by attributing them to patch-level SAE activations. The authors quantitatively shows better in interpreting concepts.

Critical strengths
  • The paper addresses a meaningful limitation of interpreting SAE latents using only top activating examples
  • The authors attempt to quantitatively evaluate the additional information provided by intervention reconstructions
Critical weaknesses
  1. The practical benefit of intervention reconstruction is not sufficiently established.
  • Although the ambiguity of top activating examples is a valid concern, it is difficult to grasp how severe this problem is across the SAE dictionary. Without this information, the reviewer cannot agree that the problem is significant
  • The improvements from intervention reconstruction in Table 1 appear relatively marginal, mostly around 1–2 percentage points compared to top-bottom, raising questions about their practical significance. Moreover, it is unclear whether these gains come from intervention-specific information or simply from providing additional inputs. Comparing against additional top/bottom examples under a matched input budget would help clarify its contribution.
  • The proposed benchmark measures whether an LLM can identify which image has higher SAE activation, but it is unclear whether improved activation prediction accuracy necessarily reflects a more accurate interpretation of the visual concepts represented by the latent. Is the benchmark properly designed so that humans can also achieve reasonable scores? Even a small-scale human evaluation of whether intervention reconstructions help distinguish the visual concepts represented by ambiguous features would provide more direct evidence.
  • The reported improvements are relatively small, but the paper does not establish whether they are statistically significant or robust to different samples of SAE features and image pairs. Confidence intervals and evaluations across different random samples would help establish the reliability of the results.
  1. The [CLS] intervention experiments do not sufficiently support the paper's main interpretability objective.
  • While the experiments show improved activation control and provide qualitative analyses of contributing patch-level features, it remains unclear whether these results lead to more accurate semantic interpretations of [CLS] features. Since the paper's primary objective is clear latent interpretation, directly evaluating whether intervention reconstructions help identify the visual attributes represented by [CLS] features would better support its central claim. Also the visual differences in examples are too small, and it is unclear whether they really matter to human interpreters.
Questions for authors
  • How widespread is the ambiguity of top activating examples across SAE latents? Can the authors estimate how frequently this issue arises beyond the selected qualitative examples?
  • Could additional top/bottom examples or carefully selected contrastive examples (exclusively activating samples) resolve similar ambiguities? Does intervention reconstruction still provide a benefit when compared against these alternatives under a matched visual input budget?
  • How well does the proposed activation prediction benchmark reflect semantic interpretability? Even a small-scale human evaluation comparing latent interpretations with and without intervention reconstructions would help validate this connection.
  • Are the improvements in Table 1 statistically significant and robust to different random samples of SAE features and image pairs?
  • How does the proposed sparse steering method improve the semantic interpretation of SAE latents?
  • How do the authors distinguish meaningful visual changes associated with the target SAE latent from artifacts introduced by the RAEv2 decoder?
5 comments
AuthorsOpenReview ↗

Author Response (1/2)

Dear Reviewer aEZ9,

We sincerely thank the reviewer for the constructive feedback and for recognizing the significance and the quantitative experiments in our paper. Our responses to the questions are listed below.

W1. Practical benefit of intervention reconstruction

W1.1/Q1. Ambiguity of top-activating examples across SAE latents

We thank the reviewer for this question. The ambiguity of top-activating examples is particularly relevant to fine-grained domains. We find that a typical SAE feature on NABirds (DINOv3) is activated by 108 of 404 categories (27%), compared with 58 of 1,000 categories (6%) on ImageNet. This suggests SAE activations are distributed across many categories, and the semantics are more difficult to infer based on the category label. Furthermore, the median SAE feature activates on only 2.7% of image patches for NABirds, indicating that the features might be very localized. The difficulty is also reflected in our quantitative benchmark: using top-activating examples alone achieves 66.3%, 64.3%, and 67.4% interpretation accuracy on NABirds, CUB, and Cars, respectively, compared with 74.7% on ImageNet.

While these results characterize the difficulty of interpreting fine-grained SAE features, directly quantifying the prevalence of ambiguity would require reliable independent interpretations of individual features. Our intervention reconstruction provides complementary visual evidence by modifying the target feature, helping distinguish its effects from co-occurring attributes. Our experiments show consistently improved interpretation accuracy on fine-grained datasets.

W1.2/Q2. Practical significance of intervention reconstructions

We thank the reviewer for this suggestion and present the results below.

Explanation budget

We first increase the number of image–overlay pairs to 10 for top, so the total number of images matches the 20 images for top + overlay + intervention. Similarly, we increase the number of explanation examples to 9 for top + bottom, resulting in 27 images in total vs. 25 images with intervention reconstructions. We report the results in the table below.

ExplanationOriginalMore examples+ Intervention
Top66.366.068.2
Top + Bottom73.073.374.8

Increasing the number of examples has a marginal influence on the prediction accuracy. These results suggest that the benefit of intervention explanations cannot be explained simply by providing more examples.

We have added the explanation budget ablation study to Appendix D.2.

Contrastive examples

We further investigate whether selecting more discriminative examples for explanation can resolve ambiguities. We use the same approach of constructing the blind benchmark to select top–bottom pairs, but from the training set. The resulting top and bottom examples have high overall similarity, but differ in the target dimension. However, the accuracy on NABirds is 71.3%, below the 73.0% achieved with standard top- and bottom-activating examples. This suggests that selecting examples with stronger activation contrasts does not necessarily produce more informative explanations. The main reason is that the image pairs that are overall similar often cannot create the sharpest contrasts in the target dimension. Comparatively, interventions can modify individual dimensions, and thereby create more targeted contrasts to help interpret sparse features.

We have added the contrastive-explanation experiment results to Appendix D.3.

W1.3/Q3. Human study

We thank the reviewer for this suggestion and have run a human study with the same protocol as the LLM judge. Participants saw the explanation for a latent and a held-out image pair, and chose which image activates the latent more strongly; they could also abstain with “Hard to tell”. We collected 360 responses over 259 latents.

ExplanationCorrectIncorrectHard to tell
Top examples71 (59.2%)15 (12.5%)34 (28.3%)
Top + bottom82 (68.3%)13 (10.8%)25 (20.8%)
+ Intervention reconstructions84 (70.0%)20 (16.7%)16 (13.3%)

When participants chose an image, they were correct 81–86% of the time in every condition, far above the 50% chance level. The results indicate that humans can produce meaningful evaluations for this benchmark. Their errors also coincide with those of the LLM judge: the judge is correct on 71–82% of the pairs humans answered correctly, but only 38–65% of the pairs they answered incorrectly.

The number of correct answers increases from top examples (71) to top + bottom examples (82) to intervention reconstructions (84), the same ordering the LLM judge shows on the full benchmark (66.3% → 73.0% → 74.8%). We will extend the study with more participants and test dimensions.

We have added the human study results to Appendix E.1.

AuthorsOpenReview ↗

Author Response (2/2)

W1.4/Q4. Statistical significance

We assess statistical uncertainty using dimension-level bootstrapping with 20,000 replicates. We resample SAE dimensions while retaining their associated image pairs, which were selected to satisfy the benchmark constraints and maximize the target feature's activation contrast. The confidence intervals are reported below.

DatasetBackboneTopTop + InterventionTop + BottomTop + Bottom + InterventionGain
NABirdsDINOv366.3 [64.7, 67.9]68.2 [66.6, 69.7]73.0 [71.5, 74.5]74.8 [73.4, 76.3]+1.80 [+0.66, +2.92]
NABirdsSigLIP 261.0 [59.3, 62.7]63.8 [62.2, 65.4]67.9 [66.4, 69.4]69.2 [67.7, 70.6]+1.26 [+0.14, +2.40]
CUBDINOv364.3 [62.7, 65.9]65.4 [63.8, 67.0]70.3 [68.9, 71.8]71.4 [70.0, 72.8]+1.08 [-0.06, +2.22]
CUBSigLIP 263.1 [61.5, 64.7]64.8 [63.3, 66.3]69.7 [68.3, 71.2]71.1 [69.6, 72.5]+1.30 [+0.22, +2.40]
CarsDINOv367.4 [65.9, 68.8]68.4 [67.0, 69.8]73.2 [71.8, 74.5]74.3 [72.9, 75.6]+1.08 [-0.06, +2.23]
CarsSigLIP 265.5 [64.0, 67.0]66.0 [64.5, 67.5]69.4 [67.9, 70.8]71.0 [69.6, 72.4]+1.60 [+0.50, +2.69]
ImageNetDINOv374.7 [73.3, 76.1]71.4 [69.9, 72.8]76.9 [75.6, 78.2]76.2 [74.9, 77.5]-0.70 [-1.74, +0.36]
ImageNetSigLIP 271.8 [70.4, 73.3]72.3 [70.8, 73.7]73.5 [72.1, 74.9]73.9 [72.5, 75.3]+0.34 [-0.64, +1.32]

Adding interventions to Top + Bottom improves interpretation accuracy across all six settings on the fine-grained NABirds, CUB, and Cars datasets, with an average gain of 1.4. Four improvements have 95% confidence intervals excluding zero, while two additional improvements (CUB and Cars with DINOv3) have lower bounds close to zero (−0.06 percentage points). These results provide statistical support for the benefits of intervention explanations, particularly on fine-grained datasets.

We have added the statistical significance to Appendix E.2.

W2. [CLS] explanation

W2.1/Q5. Quantitative experiments for [CLS] explanation

We thank the reviewer for this suggestion. To directly evaluate whether intervention reconstructions improve the interpretation of [CLS] SAE features, we constructed a similar paired benchmark on NABirds containing 1,000 blind image pairs across 200 [CLS] features.

Explanation MethodAccuracy (%)
Top57.5
Top + Interventions58.5
Top + Bottom61.1
Top + Bottom + Interventions62.1

Using intervention reconstructions improves both baselines by 1.0%. Notably, on 190 pairs where top-example and top–bottom explanations yield different predictions, interventions improve accuracy from 40.5% to 51.1%. These results suggest that intervention reconstructions provide complementary visual evidence for interpreting [CLS] features, particularly when top- and bottom-examples are ambiguous.

W2.2. Small visual differences in [CLS] explanation

The subtle visual differences are partly attributable to the fine-grained nature of the evaluated domains, where meaningful attributes often involve small variations in morphology, coloration, or texture. Moreover, the magnitude and semantic coherence of the reconstructed changes depend on the underlying SAE features. Thus, subtle or ambiguous changes may also reflect limitations of the learned SAE representations, instead of the intervention method itself. Our method aims to reveal the visual information encoded by existing SAE features, including fine-grained attributes whose effects may be difficult to perceive. Using the resulting visual changes as an approach to evaluate the quality of SAE representations can also be an important future direction.

We have included the experimental results and discussion in Table 3 in Section 4.3.

Q6. Meaningful visual changes vs. reconstruction artifacts

We thank the reviewer for this question. Our intervention analysis directly compares two reconstructed images generated by decreasing and increasing the target SAE feature activation. Both images are produced using the same RAEv2 decoder, with interventions targeting localized visual changes while preserving the remaining image content. This provides a direct contrast that helps distinguish feature-associated changes from shared reconstruction artifacts. Although this comparison cannot eliminate decoding artifacts, visual changes across intervention strengths provide direct evidence to associate with the target SAE feature.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts, and I believe the paper has improved. However, some concerns remain.

First, while the additional experiments provide stronger evidence for the benefits of intervention reconstructions, the improvements remain relatively small, typically around 1–2 percentage points. In the human study, adding interventions slightly increases correct responses, but also increases incorrect responses. This raises questions about whether the method meaningfully improves interpretation reliability.

Second, in W1.1, the authors state that "the ambiguity of top-activating examples is particularly relevant to fine-grained domains." However, it remains unclear whether the underlying foundation models and SAEs reliably capture meaningful fine-grained attributes in the first place. As the authors acknowledge in W2.2, subtle visual changes may reflect limitations of the learned SAE representations. If the foundation models or SAEs fail to learn these fine-grained attributes, it is unclear how much practical value intervention reconstructions can provide in interpreting them.

Reviewer_TnSmOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a framekwork, RECSAEI, for performing steering and reconstruction of interpretable SAE features to better understand what those features are capturing. They provide methods for both patch and CLS tokens that allow one to connect the decision of the CLS token to the concepts found in the patch token SAE dictionary.

They evaluate their method on the ability to generate accurate feature explainations, the ability for CLS feature steering, and also the cross-domain stability of feature editing.

Critical strengths

The paper is very well written and the motivation of finding better, more causal features in models is great.

The cross-image consistency is a really cool experiment (although purely qualitative).

Critical weaknesses

The core weakeness I have is that the paper is missing references to prior work and does not situate it's contributions appropriately. Many works over the last 7 years have explained visual classifiers by reproducing counterfactual inputs while steering interpretable concepts in order to better distinguish what the concept encodes [a,b,c,d]. Additionally, the cited Daujotas (2024b) already intervenes on visual SAE features and decodes the results to test and disambiguate feature interpretations. This is not to mention the rich work on language in this front where the semantic and causal efficacy of features validated via steering and causal interventions has a long history [e,f,g]. Moreover, the gradient attribution/saliency literature is extremely vast too [h] so I am not sure about the contributions there, other than performing attribution from the CLS token to intermediate visual tokens rather than the first layer tokens themselves. Thus, my argument is that the current paper needs to position itself with respect to these works much better. I believe the novel contributions of this work surround the specific implementation details of using an RAE and connecting the CLS to patch tokens through gradient attribution and it was strange to me to not see a single baseline compared to for steering-based interpretability methods (e.g., [c] or [d] could potentially be adapted to target SAE activations). Overall, a clear contribution section needs to exist and outline the core novel contributions that differentiates itself.

[a] Ghandeharioun et al. DISSECT: Disentangled Simultaneous Explanations via Concept Traversals (ICLR 2022)

[b] Lang et al. Explaining in Style: Training a GAN to Explain a Classifier in StyleSpace (ICCV 2021)

[c] Goetschalckx et al. GANalyze: Toward Visual Definitions of Cognitive Image Properties (ICCV 2019)

[d] Singla et al. Explanation by Progressive Exaggeration (ICLR 2020)

[e] Geiger et al. Causal Abstractions of Neural Networks (NeurIPS 2021)

[f] Geiger et al. Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability (JMLR 2025)

[g] Templeton et al. Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet (Anthropic Blog Post, 2024)

[h] Kasmi et al. One Wave To Explain Them All: A Unifying Perspective On Feature Attribution (ICML 2025)

Questions for authors

What are the exact contributions made in this paper, particularly when situated against the related work [a-h] above? I would be happy to be convinced that the novelty of this work (e.g., related to the CLS - Patch token connection) is sufficiently novel and interesting for publication but currently do not think it is and have a difficult time assessing the exact novelty here.

7 comments
AuthorsOpenReview ↗

Author Response (1/3)

Dear Reviewer TnSm,

We sincerely thank the reviewer for the constructive feedback and for recognizing the clarity and motivation of our paper. Our responses to the questions are listed below.

W1/Q1. Contribution and differences from prior work

We thank the reviewer for highlighting these relevant works. We will address the concerns point by point.

W1.1. Comparison with counterfactual explanation methods

Our contribution is to apply spatially localized interventions to SAE features learned directly from pretrained vision encoders, enabling investigation of internal visual representations beyond classifier-oriented explanations.

SAEs provide a holistic view of internal representations

Previous methods primarily investigate visual transformations that influence classifier predictions or target image properties through image-level generative latent spaces. In contrast, SAE features are learned by decomposing the encoder's internal representations in an unsupervised manner. These features can capture recurring visual properties across observations, including attributes that are not necessarily discriminative for a particular classification task. This is particularly valuable for fine-grained domains, where subtle traits and local patterns may vary across undefined categories. Interpreting such features could provide a common basis for characterizing and comparing meaningful variation across observations.

Our approach enables targeted, localized visualization of SAE features

Although previous approaches can reveal meaningful directions, they do not necessarily associate with SAE features and often operate on image-level representations. Less targeted, localized interventions may even introduce new ambiguity in interpreting sparse features. Our framework addresses this gap by intervening on individual patch-level SAE features at selected spatial locations and reconstructing their visual effects. This enables us to investigate how learned features manifest across different image regions and visual contexts, supporting a more fine-grained understanding of the representations underlying observations.

Methodological differences

  • [a, b]: DISSECT and StylEx identify interpretable attributes in generative latent spaces, guided by their relevance to classifier predictions. Our work investigates sparse features extracted directly from the internal representations of pretrained vision encoders. SAE dimensions correspond to features of the encoder itself, instead of a particular feature direction identified by classifiers, allowing us to understand how visual information is represented within the model beyond what features are important for classification.
  • [c]: GANalyze learns directions in a GAN’s latent space to manipulate properties such as image memorability. It operates on GAN-generated images instead of user-given real images. Our method directly reconstructs interventions starting from representations of real input images.
  • [d]: Progressive Exaggeration generates counterfactual images by traversing the prediction probabilities of a binary classifier. Directly extending this formulation to interpret individual SAE dimensions would require feature-specific training on tens of thousands of SAE dimensions.

Daujotas (2024b) uses diffusion models to recover the intervention on SAE features, but still relies on image-level features. The examples shown in the blog generally have large visual changes involving multiple concepts. While it explains concepts such as image style and object presence, it will be less straightforward when applied to visualizing the fine-grained concepts targeted in our paper.

AuthorsOpenReview ↗

Author Response (2/3)

W1.2. Comparison with intervention methods in language models

We agree that the general principles of intervention-based interpretability apply across modalities, and we do not claim novelty in proposing intervention. Our contribution is to enable image-space visualization of sparse feature interventions.

Compared with language models, visual representations present a distinct challenge in interpreting intervention effects. Language models have a native decoding mechanism that allows changes in internal representations to be examined through generated text. In contrast, pretrained discriminative vision encoders generally lack an image decoder, making it difficult to directly visualize what changes in the input space correspond to interventions on individual features. This challenge is particularly important for fine-grained visual domains, where feature semantics may involve subtle traits and localized visual patterns that are difficult to characterize through activation changes or downstream predictions alone. Our work addresses this challenge by enabling image-space reconstruction of interventions on visual SAE features.

W1.3. Comparison with gradient attribution methods

We would like to clarify that we do not claim methodological novelty in gradient attribution itself. Our contribution lies in using gradient-based sensitivity to identify sparse patch-level SAE features that support a target [CLS] feature, and leveraging this decomposition for more effective interventions.

Importantly, our experiments show that sparse attribution produces more stable visual changes and more consistent responses in the target [CLS] feature compared with direct dense gradient steering. Furthermore, the identified patch-level SAE features can exhibit distinct activation patterns and intervention effects, providing additional insights into how localized representations contribute to image-level features. Thus, our emphasis is on the effectiveness and interpretability of sparse feature interventions.

W1.4. Use of RAE

Regarding the use of RAE, our choice is motivated by the need to reconstruct interventions on patch-level representations of pretrained vision encoders. Existing approaches such as GANalyze and Progressive Exaggeration primarily operate through image-level generative representations and do not directly support reconstruction from patch embeddings. RAE provides a feasible mechanism for mapping spatial patch representations back to images, allowing us to visualize localized feature interventions. Our framework is not restricted to RAE and could also incorporate other generative decoders that support the corresponding patch representations.

W1.5. GANalyze baseline comparison

We implemented GANalyze-style steering as a baseline: for each of 200 SAE dimensions from ImageNet–DINOv3, we trained a BigGAN-deep latent direction to increase or decrease the dimension’s foreground activation, then placed the resulting reduced/increased image pairs as the intervention explanation. GANalyze reaches 75.0% prediction accuracy, compared with 76.5% for RecSAEI. We have also added a qualitative comparison between the edited images from GANalyze and RecSAEI. In Fig. 9, we observe that GANalyze may modify the whole image, including pose and background. It also introduces more generative artifacts. Comparatively, RecSAEI edits are confined to the patches where the feature is active.

GANalyze is also far more expensive. Fitting one latent direction takes 500 optimization steps through BigGAN and the encoder, about 4 minutes per dimension (5 example images) on a V100, whereas a RecSAEI edit is a single decoder pass, about 1 second per dimension on the same GPU.

We have added the baseline comparison to Table 2 and Figure 9 in Section 4.3.

AuthorsOpenReview ↗

Author Response (3/3)

W1.6. Summary of contributions

We summarize the main contributions as follows:

  • We develop Reconstructive SAE Intervention (RecSAEI), which reconstructs visual effects of interventions on SAE features within pretrained vision encoders. By applying and reconstructing interventions on patch representations, our approach enables more localized, targeted counterfactual generation for sparse model features.

  • We introduce a sparse attribution approach that identifies patch-level SAE features associated with a target [CLS] SAE feature. Intervening on these components produces more stable visual changes and consistent target-feature responses compared with direct dense gradient steering.

  • We evaluate RecSAEI on fine-grained visual datasets, examining its interpretability and the effectiveness in controlling target [CLS] SAE features. We show that intervention reconstructions provide essential complementary information in interpreting sparse features.

We have added the contribution summary to the end of the introduction and have added the discussion above to a dedicated section in Appendix A. We have also revised the abstract, introduction, and related work to provide more explicit positioning of this paper.

Reviewer_An8FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a method to apply causal interventions for feature interpretability of SAEs trained on image models, in a way aking to how its done with language models. The original idea is to steer the base model, then reconstruct the result via the RAE decoder. The only obstruction, how to map interventions on [cls] features, is dealt with using gradient attribution.

Critical strengths

The method is original, and fills an important gap in mechanistic interpretability for image models, namely, the difficulty of performing causal interventions to identify features.

The reviewer appreciates the clarity of the exposition.

Critical weaknesses

Evaluation is the weakest point of the manuscript. Table 1 shows modest gains for the proposed methods, but without statistical significance tests it is hard to know whether this is just an artifact. For example, it might be that top+bottom+intervention improves over top+bottom because more explanations are provided. Further, a quantitative evaluation on the interpretation of confounded dimensions might be helpful.

Questions for authors

Why use a SAE trained on final-layer [CLS] representations and a source SAE trained on penultimate-layer patch representations? Why not the same layer?

Line 302, you claim “bottom examples cannot directly disambiguate confounded features”. What do you mean? Does not the performance when using both top and bottom explanations hints that this is what happens?

4 comments
AuthorsOpenReview ↗

Author Response (1/2)

Dear Reviewer An8F,

We sincerely thank the reviewer for the constructive feedback and for recognizing the significance and clarity of our paper. Our responses to the questions are listed below.

W1. Quantitative evaluations

Thank you for the suggestion. We have listed additional quantitative evaluation results below and in the paper revision.

W1.1. Statistical significance

We assess statistical uncertainty using dimension-level bootstrapping with 20,000 replicates. We resample SAE dimensions while retaining their associated image pairs, which were selected to satisfy the benchmark constraints and maximize the target feature's activation contrast. The confidence intervals are reported below.

DatasetBackboneTopTop + InterventionTop + BottomTop + Bottom + InterventionGain
NABirdsDINOv366.3 [64.7, 67.9]68.2 [66.6, 69.7]73.0 [71.5, 74.5]74.8 [73.4, 76.3]+1.80 [+0.66, +2.92]
NABirdsSigLIP 261.0 [59.3, 62.7]63.8 [62.2, 65.4]67.9 [66.4, 69.4]69.2 [67.7, 70.6]+1.26 [+0.14, +2.40]
CUBDINOv364.3 [62.7, 65.9]65.4 [63.8, 67.0]70.3 [68.9, 71.8]71.4 [70.0, 72.8]+1.08 [-0.06, +2.22]
CUBSigLIP 263.1 [61.5, 64.7]64.8 [63.3, 66.3]69.7 [68.3, 71.2]71.1 [69.6, 72.5]+1.30 [+0.22, +2.40]
CarsDINOv367.4 [65.9, 68.8]68.4 [67.0, 69.8]73.2 [71.8, 74.5]74.3 [72.9, 75.6]+1.08 [-0.06, +2.23]
CarsSigLIP 265.5 [64.0, 67.0]66.0 [64.5, 67.5]69.4 [67.9, 70.8]71.0 [69.6, 72.4]+1.60 [+0.50, +2.69]
ImageNetDINOv374.7 [73.3, 76.1]71.4 [69.9, 72.8]76.9 [75.6, 78.2]76.2 [74.9, 77.5]-0.70 [-1.74, +0.36]
ImageNetSigLIP 271.8 [70.4, 73.3]72.3 [70.8, 73.7]73.5 [72.1, 74.9]73.9 [72.5, 75.3]+0.34 [-0.64, +1.32]

Adding interventions to Top + Bottom improves interpretation accuracy across all six settings on the fine-grained NABirds, CUB, and Cars datasets, with an average gain of 1.4. Four improvements have 95% confidence intervals excluding zero, while two additional improvements (CUB and Cars with DINOv3) have lower bounds close to zero (−0.06 percentage points). These results provide statistical support for the benefits of intervention explanations, particularly on fine-grained datasets.

We have added the statistical significance to Appendix E.2.

W1.2. Explanation budget

We investigate whether the improvement simply comes from providing more image examples. Specifically, we increase the number of image–overlay pairs to 10 for top, so the total number of images matches the 20 images for top + overlay + intervention. Similarly, we increase the number of explanation examples to 9 for top + bottom, resulting in 27 images in total vs. 25 images with intervention reconstructions. We report the results in the table below.

ExplanationOriginalMore examples+ Intervention
Top66.366.068.2
Top + Bottom73.073.374.8

Increasing the number of examples has a marginal influence on the prediction accuracy. These results suggest that the benefit of intervention explanations cannot be explained simply by providing more examples.

We have added the explanation budget ablation study to Appendix D.2.

W1.3. Interpretation of confounded dimensions

We evaluate whether intervention explanations can help distinguish SAE dimensions with overlapping activation patterns. Specifically, we curated 120 pairs of SAE dimensions with patch activation cosine similarity ≥ 0.5 on the same reference images. We measure the semantic similarity between descriptions of confounded dimensions and compare using only top examples and adding interventions. Using Qwen3-Embedding-0.6B, intervention reconstructions reduce the mean cosine similarity between generated descriptions for paired dimensions from 0.87 to 0.81. We observe consistent trends across other embedding models, such as BGE-large.

Embedding modelConfounded similarity ↓Repeat-vs-confounded AUROC ↑
Qwen3-Embedding-0.6B0.87 → 0.810.59 → 0.71
BGE-large0.88 → 0.850.62 → 0.71

We further evaluate whether descriptions of the same dimension (independent repetitions) can be distinguished from those of different, potentially confounded dimensions using text-embedding similarity as a classification score. The resulting AUROC measures how reliably descriptions from the same dimension receive higher similarity scores than cross-dimension description pairs. We observe significantly improved AUROC from both encoders. These results suggest that interventions provide additional evidence that helps differentiate otherwise similar feature interpretations.

We have added the results and discussion to Appendix E.3.

AuthorsOpenReview ↗

Author Response (2/2)

Q1. The layer used for [CLS] and patch SAE representations

The final-layer [CLS] token aggregates information from the penultimate-layer patch representations through the final transformer block. Therefore, the penultimate-layer patch embeddings can be used to identify patch-level features that contribute to the final [CLS] representation. Oppositely, the last-layer patch representations do not contribute to the [CLS] representations.

Q2. Clarification of improvements from bottom examples

Thank you for pointing out this imprecise statement. Bottom examples provide useful contrastive information and can help disambiguate confounded features. However, top and bottom examples may differ in multiple visual attributes. Comparatively, interventions provide a more targeted comparison by modifying the target feature within the same image context. We have revised the claim to “While bottom examples provide useful contrastive information, interventions offer more targeted evidence for distinguishing potentially confounded features.”

I appreciate the author's detailed response. In particular, the explanation budget experiment is clarifying. However, I still remain unconvinced that interventions in the visual domain are fundamental for achieving high quality explanations. In this particular experiment, I believe results are not conclusive, because the images that are being added may not provide 'new information'. The explainer model is helped by the divergence between top and bottom, more than anything else. I thank the authors for the addition of confidence intervals, that show that interventions indeed help achieve better explanations. But these gains are modest, and it still might be the case that interventions help the explainer model by providing more variability to the information in 'top' samples. I would be interested in seeing the following experiment: increase the explanation budget while maximizing the diveristy of the samples. For example, sample from top 10th percentile (resp bottom), then within each group select a subset of samples that maximizes diversity. This would help clarify whether we actually need causal interventions.

I've taken a look at the revised manuscript, and appreciate the changes added. But for now, my score remains unchanged.

Dear Reviewer An8F,

Thank you for this suggestion. We have run the proposed experiment. For each of the 1,000 NABirds patch SAE dimensions, we took the top 10% of activating images and selected the subset that maximizes diversity. We then used the same increased budget (10 image–overlay pairs for Top, and 9 image–overlay–bottom sets for Top + Bottom) and tested the prediction accuracy.

InputImagesAccuracy
Top-51066.3
Top-102065.9
Diversity-max 102059.5
Top-5 + interventions2068.2
Top-5 + bottom1573.0
Top-9 + bottom2773.3
Diversity-max 9 + bottom2765.5
Top-5 + bottom + interventions2574.8

The diverse set yields more than 2×2\times the cosine distance (1−1- cosine similarity) of the top-activating examples selected in our benchmark. However, these diverse top examples significantly decrease the prediction accuracy.

There are two potential reasons:

  • While having high activation, the top 10% images still represent the target feature more weakly than the actual top-5 or top-10 images selected in our benchmark
  • Diverse examples may also introduce more differing factors. In contrast, the proposed intervention changes only the target feature in a fixed image, isolating the factor that additional observational samples may leave confounded.

We also note that the proposed intervention does not introduce large variability into the explanation inputs compared with the diverse samples. On the contrary, it produces very targeted changes: each edited image differs from its original only in what the feature controls. This is why interventions carry the most direct information about the semantics of the corresponding SAE dimension, and why adding diverse real samples or increasing the budget cannot substitute for them. We have added this experiment to Appendix D.2.

Comments on the paper

1 comment

We thank all reviewers and the AC for their efforts and constructive feedback and suggestions. We have revised the manuscript to clarify our contributions, strengthen the experimental evidence, and better position our work within the existing literature. The major revisions are summarized below.

  1. Clarified contributions and connections to prior work (Abstract, §§1–2, Appendix A). We have expanded our discussion of prior counterfactual generation, visual SAE interventions, and gradient attribution methods. Our contributions primarily focus on reconstructing localized interventions on patch-level SAE features and decomposing [CLS] features into supporting patch-level components, emphasizing their relevance to interpreting recurring visual patterns in fine-grained domains.

  2. Strengthened the evaluation of intervention-based feature interpretation (§§4.1, 4.3, Appendices D.2–D.3, E.1–E.3). We have added comparisons with GAN-based counterfactual generation, controls for explanation budgets and contrastive examples, and a new evaluation of intervention reconstructions for interpreting [CLS] SAE features. We have also introduced experiments examining the disambiguation of confounded SAE dimensions, a human study, and statistical analyses with confidence intervals to further validate the benefits of intervention-based explanations.

  3. Expanded the analysis of sparse [CLS] intervention (Appendix D.6). We have introduced comparisons with alternative SAE architectures (TopK and Matryoshka), PCA, and direct dense steering to investigate the role of source feature dictionaries. The results demonstrate that SAE-based representations identify influential source dimensions more effectively with fewer components, supporting their use for sparse attribution and targeted [CLS] intervention.

We hope that these revisions have adequately addressed the reviewers' concerns. We sincerely appreciate the reviewers' constructive feedback, which has helped us improve the manuscript. We would be happy to address any further questions or concerns before the discussion deadline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.