- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper proposes SharedSAE, a single sparse autoencoder trained on the residual updates of four SDXL 1.0 attention blocks across all 50 denoising steps. Previous diffusion SAEs mostly use one dictionary per block, or one per block over time, so concepts end up in separate feature spaces. The authors exploit SharedSAE to do unlearning and they also use the dictionary to trace how concepts move and evolve through the network.
Critical strengths
- SharedSAE addresses the problem of several SAEs at different layers (this means a higher number of trainable parameters and SAEs to store). It also acts as an SAE analysis tool, showing how concepts move across blocks and time.
- Unlearning results are promising.
Critical weaknesses
- The proposed method does not compare with any SAE-based unlearning method (SAeUron, SAEmnesia etc.), and are not evaluated for fidelity or robustness (adv attacks). This should motivate whether a shared multi-block dictionary improves on the simpler per-block or single-block SAE approach it is meant to replace.
- The paper argues that one shared dictionary lets features be "compared and reused directly across the covered positions." (line 151), but no experiment checks that a feature means the same thing in different blocks (see also questions).
Questions for authors
- I was expecting the proposed method to mitigate the feature-splitting problem across different blocks, i.e., to have the same feature IDs for the same concepts across different blocks. But Figure 6b shows the opposite; can you explain why?
- Which blocks are edited during erasure (is B all four, or task-dependent), and how does erasure quality change when intervening on a single block versus all four?
- What about all the other blocks in the model?
- How were λ_erase = 10 and 5 selected, and how do erasure success and image quality (e.g., FID/CLIP score on unrelated prompts) vary with λ?
- Why rank-16 low-rank adapters?
- Can the method be applied to other diffusion models? -What's the difference between crosscoders and sharedSAEs? When should you use one instead of the other?
3 comments
Response to Reviewer Dcfj (2/3)
Q1 and W2. Cross-block semantic consistency and feature splitting
However, we would like to emphasize that sharing a dictionary across layers does not necessarily imply that a given feature must remain active at every layer.
Lawson et al. (Residual Stream Analysis with Multi-Layer SAEs, ICLR 2025) observed that although features learned by multi-layer SAEs can exhibit activation across layers when aggregated over many tokens, their activations for an individual token are often concentrated in only a few layers rather than uniformly distributed throughout the network.
Different blocks perform different computations. The same concept can vary in form, granularity, and activation time, so forcing identical IDs could hide block-specific differences.
Of course, layer-selective activation and feature splitting are not identical phenomena. Although SharedSAE employs a shared dictionary, the same semantic concept may still be decomposed into multiple sparse directions. Further reducing such feature splitting while preserving reconstruction quality and interpretability remains an important research question.
The main additional evidence is provided in Appendix A.2 (p. 16, Figure 10).
Using a generated dog as an example, Figure 10 analyzes 28 feature directions corresponding to seven semantic parts across four blocks. Although the feature IDs associated with the same semantic part are not necessarily identical across blocks, their activation maps exhibit clear spatial correspondence. This allows us to identify and compare representations of the same semantic part at different network positions.
- Appendix A.3 (p. 17, Figure 11): Examines the responses of fixed feature IDs across different generated images, demonstrating semantic reuse across input instances.
- Appendix A.4 (p. 18, Figure 12): Investigates the temporal evolution of semantically related features across blocks, revealing differences in when the same semantic parts become active during denoising.
- Appendix B.1 (p. 20, Figure 14): Uses bicycle wheels as an example to further analyze cross-block spatial semantic correspondence and temporal alternation.
2. Blocks Edited During Concept Erasure and the Effects of Single-Block versus Multi-Block Intervention
In all SDXL concept-erasure experiments, we consistently intervene in four attention blocks: down.2.1, mid.0, up.0.0, and up.0.1, across all 50 denoising steps.
See Section 3.2 (p. 5) and Appendix C.4 (p. 26).
Although the four blocks share the same SAE dictionary, each block uses its own concept-related feature set, identified from its corresponding activation statistics. In our implementation, the residual updates from all four blocks can be collected during a single U-Net forward pass at each denoising step, without requiring separate forward passes for individual blocks. This facilitates unified feature collection and analysis across network positions.
This behavior is consistent with the observation that the same semantic concept may have complementary representations across different blocks. When only one network position is modified, concept-related information retained in other blocks can still influence the final generated image.
3. Network Blocks Not Covered by SharedSAE
We prioritize blocks near the U-Net bottleneck to study higher-level semantics. Shallower and output-proximal layers tend to focus more on local details such as textures and edges.
Second, we aim to preserve the generative capabilities of the remaining network components. Concept erasure does not require indiscriminately modifying every representation associated with a target concept. Instead, we apply targeted interventions at a limited number of positions, allowing the remaining blocks to continue contributing to background generation, composition, and non-target visual content.
Accordingly, blocks not covered by SharedSAE retain their original computations without additional modifications. This is a deliberate choice to limit the intervention scope and avoid unnecessary degradation of image quality.
Regarding heterogeneous hidden dimensions, the four selected blocks currently share the same residual dimension, d = 1280, allowing them to directly use a common dictionary. In principle, the framework could be extended to blocks with different dimensions by using block-specific input projections into a common representation space and corresponding output projections back to the original dimensions. Spatial alignment or pooling could also be introduced when blocks operate at different spatial resolutions.
During our initial exploration, we found that intervening in the four selected blocks already produced effective concept-erasure results. We therefore adopted this relatively simple configuration rather than introducing additional architectural complexity.
Response to Reviewer Dcfj (3/3)
4. Selection of λ_erase = 10 and 5 and Their Effects on Generation Quality
We use λ_erase = 10 as the default setting for general object and style concept erasure. This value was selected empirically based on preliminary experiments rather than through an exhaustive hyperparameter search.
We believe that this choice is also related to differences in the objectives of various erasure tasks. For instance, erasing object concepts such as cars or dogs generally requires substantial changes to the subject's shape and semantic identity. In contrast, nudity erasure does not require removing the person; instead, it primarily aims to suppress visual attributes associated with nudity. Using a smaller value of λ_erase = 5 allows the model to generate clothing or other contextually appropriate alternatives along the subsequent denoising trajectory.
As reported in Section 4.5 (p. 9) and Table 1 (p. 6), using λ_erase = 5 reduces the number of NudeNet-positive generations on the full I2P benchmark from 225 to 12 out of 4,703 prompts. Upon manual inspection, none of these 12 remaining flagged images exhibited obvious nudity-related content.
On 5,000 COCO prompts (Table 9, p. 32), FID changes from 25.02 to 25.10, while CLIP Score changes from 32.01 to 31.97. These results indicate that overall generation quality and text-image alignment are largely preserved under the chosen intervention strength.
5. Rationale for Using Rank-16 Low-Rank Adapters
Rank 16 is an empirical configuration adopted in our current implementation to balance block-specific adaptation capacity against additional parameter overhead.
Although we have not conducted an exhaustive hyperparameter sweep over adapter ranks, we have added an ablation study that removes the block-specific adapters in Table 2 (p. 7), Figure 4 (p. 7), and Section 4.4 (p. 8).
Removing the adapters has little effect on reconstruction FVU but leaves more recognizable target remnants and local artifacts in the erasure examples (Table 2, p. 7 and Fig. 4, p. 7). We evaluated adapter removal without a sweep over ranks.
6. Applicability to Other Diffusion Models and Differences from Crosscoder
See Appendix F.1 (Figures 41–43, upload pp. 34–36, full pp. 41–43).
Specifically, we apply SharedSAE to transformer blocks 6–20 of PixArt-α 512, following the same shared sparse representation, concept localization, and erasure framework, while retraining the SAE for the corresponding architecture.
First, they differ in their reconstruction targets. A typical Crosscoder jointly models hidden-state activations from multiple layers or models to learn sparse features shared across representation positions. In contrast, SharedSAE reconstructs the residual update introduced by each U-Net block:
x_b = h_out,b − h_in,b
This definition is given in Section 3.2 (p. 5, Eq. (1)).
Decomposing each residual update into sparse directions lets us study how the update influences the denoising velocity field through subsequent network computations.
See Section 3.1 (p. 4) and Appendix D (pp. 26–31).
During denoising, certain semantic attributes, spatial structures, and visual characteristics associated with a target concept may already be encoded in the current state. Even if we suppress the corresponding residual contributions, this previously established information, together with information propagated through unmodified network pathways, can still influence subsequent generation.
This may explain concept leakage and semantic substitution. After bird erasure, decorative objects can retain similar colors. After bus erasure, a yellow seat can retain the original color and aspects of the layout (Sec. 4.6, p. 10).
Second, Crosscoder and SharedSAE differ in their parameter organization and encoding mechanisms. Crosscoder typically jointly encodes activations from multiple layers and employs layer-specific decoder weights. In contrast, SharedSAE uses a single shared encoder-decoder with lightweight block-specific input and output adapters to process residual updates from different blocks, without requiring simultaneous access to all blocks' representations during inference.
During our initial investigations, we also experimented with a Crosscoder-like joint encoding scheme. Although this approach could establish shared feature IDs across different network layers, our preliminary experiments showed that the resulting features often exhibited less concentrated semantic responses, with some features responding to multiple distinct visual attributes. Their semantic interpretations were therefore less clear than those obtained using our current SharedSAE architecture.
The SharedSAE architecture is detailed in Section 3.3 (p. 5, Eqs. (3)–(4)) and Figure 3 (p. 4).
We view these approaches as complementary.
Response to Reviewer Dcfj (1/3)
Thank you for your careful reading and for recognizing the potential of SharedSAE for parameter-efficient representation, cross-block analysis, and concept erasure. Your suggestion to compare with existing SAE-based erasure methods helped us clarify what the matched baseline tests and how we evaluate erasure itself. We will release the SharedSAE training code and model weights upon acceptance.
1. Comparison with Existing SAE-Based Concept Erasure Methods
We thank the reviewer for raising this important point. We agree that comparisons with existing SAE-based methods, such as SAeUron and SAEmnesia, are valuable for a comprehensive evaluation of SharedSAE.
SharedSAE investigates whether a single shared sparse dictionary can jointly model multiple blocks and denoising timesteps while retaining useful features for semantic localization and intervention. Our training objective learns the shared representation. We apply it to concept erasure at inference time.
In fact, our concept localization procedure adopts the feature importance score introduced by SAeUron (Section 3.5, p. 6; Appendix C.2, p. 25). Thus, our main contribution is not an entirely new concept localization strategy, but rather an extension of sparse feature analysis and intervention to a shared representation framework spanning multiple blocks and denoising timesteps.
From this perspective, we believe that the most informative controlled comparison is between a single shared SAE and multiple independent SAEs covering the same network positions, under identical backbone and training conditions. Such a comparison directly isolates the benefits and trade-offs of dictionary sharing.
To this end, we have added a fair baseline consisting of four independently trained SAEs using the same SDXL 1.0 backbone, training data, and training configuration.
As reported in Tables 2 and 4 (p. 7), SharedSAE reduces the parameter count from 111.9M (the combined parameters of four independent SAEs) to 28.1M, representing a 74.9% reduction, while maintaining comparable reconstruction quality (FVU: 0.379 vs. 0.356).
More importantly, Table 3 (p. 7) shows that SharedSAE achieves higher feature localization consistency than independent SAEs under three evaluation conditions: prompt split, cross-seed, and time-window shift. The corresponding Jaccard overlaps are 0.857 vs. 0.813, 0.568 vs. 0.474, and 0.749 vs. 0.667, respectively.
For the downstream application of concept erasure, we have also added qualitative comparisons with independent single-block SAEs in Figure 4 (p. 7) and Section 4.4 (p. 8). In the presented examples, interventions using an independent single-block SAE often leave recognizable target structures, whereas SharedSAE's coordinated multi-block intervention achieves more complete and consistent concept suppression. These observations demonstrate the practical value of cross-block intervention and confirm that a shared sparse dictionary can effectively support such interventions.
For nudity erasure, SharedSAE reduces the number of NudeNet-positive generations from 225 to 12 across the full I2P benchmark of 4,703 prompts, a net reduction of 94.7% (Table 1, p. 6 and Section 4.5, p. 9). On manual inspection, none of the 12 remaining flagged outputs showed obvious nudity. This result shows that the shared dictionary can substantially suppress nudity at inference time.
Furthermore, we have added a generation-quality evaluation on 5,000 COCO prompts in Appendix E (p. 32, Table 9). After erasure, FID changes only from 25.02 to 25.10, while CLIP Score changes from 32.01 to 31.97, indicating that overall generation quality and text-image alignment are largely preserved.
Regarding direct numerical comparisons with the published erasure results of SAeUron and SAEmnesia, their underlying diffusion backbones and evaluation protocols differ from our SDXL-based setting. Directly comparing the reported scores would make it difficult to disentangle methodological differences from differences in experimental settings. In addition, SAEmnesia introduces concept-supervised training, whereas SharedSAE learns its shared dictionary without concept-specific supervision.
We believe that the newly added independent-SAE baseline more directly validates our central contribution: achieving competitive reconstruction quality, stable feature localization, and effective downstream interventions at substantially lower representation cost.