- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SAE-Audit, a white-box framework for testing concept erasure through fixed activation interventions. Sparse autoencoders identify concept-related responses that weaken after erasure, and directions derived from the original model are injected into the edited model without changing its weights or test prompts. Experiments cover six erasure methods and thirteen concepts on Stable Diffusion 1.4.
Critical strengths
-
The paper addresses an important evaluation gap by testing whether concept suppression remains robust under targeted interventions on internal activations.
-
The SAE-based framework connects changes in feature responses to recovery behavior while keeping model weights and test prompts unchanged.
-
The experiments cover six erasure methods and four concept domains, with matched random controls and alternative recovery baselines that help assess the proposed interventions.
Critical weaknesses
-
The injected directions and their values come from the original model and therefore carry target-concept information. Successful recovery could reflect bypassing a disrupted text-to-concept pathway, strengthening attenuated representations, or supplying information that the edited model no longer adequately contains. The current experiments do not distinguish these explanations. Although the paper appropriately acknowledges that recovery does not prove an intact representation survived, it remains unclear what the audit establishes about retained knowledge beyond responsiveness to externally supplied concept information. This limits its central diagnostic value.
-
The later rel experiments reused scene families whose test results had already been inspected (Appendices C.3 and J). The pilot mean-versus-median prototype decision also used test feedback, and the scene-consistency rule was finalized after inspecting eleven audits (Appendix H.1). Table 1 further selects the stronger observed test outcome across two configurations. The disclosure is welcome, but these adaptive choices introduce possible selection bias. The results provide exploratory evidence, while the reported significance does not establish performance under an independently fixed evaluation protocol.
-
Detector-level recovery does not reliably establish faithful recovery or scene preservation. Some interventions produce concept-associated textures or fragments while disrupting the requested scene. The Seurat relabeling experiment also shows substantial sensitivity to the detector’s label set. CLIP similarity alone cannot establish whether the target identity, other objects, and their relationships are preserved. A semantic assessment, potentially using a multimodal model calibrated against human judgments, would provide stronger evidence than the current similarity-based criterion.
-
The additional diagnostic insight over existing recovery attacks remains limited. Prior work already demonstrates that concepts suppressed under ordinary prompts can be recovered. The contribution therefore depends on what feature-level probing reveals beyond this established observation. However, differences across concepts and hookpoints do not clearly separate the effects of the erasure method, the concept, and the probe itself. Changing the hookpoint criterion also changes the dictionaries, selected features, prototypes, and intervention strength. Consequently, the results do not isolate why recovery succeeds or fails. The modest and nonuniform advantage over Reverse-SAeUron further leaves the added value of the proposed feature-selection rule uncertain.
-
Architectural generality is untested. All main experiments use the SD 1.4 UNet. It is unclear whether the feature-selection procedure, recovery behavior, and diagnostic conclusions extend to other backbones, such as SDXL or a DiT-based model.
Questions for authors
Could Figure 1 explicitly separate the baseline and intervention paths? Part 5 places both paths around the same central model illustration, with arrows pointing in opposite directions. This makes the generation and evaluation flow difficult to follow. Explicit labels for the two paths would clarify which computation is performed in each condition.
3 comments
Response to WdG1 (Part 2/3)
Weakness 3. Target identity and scene fidelity
The Seurat relabeling result illustrates the limitations of detector scores. Adding pointillism as a competing label reduced the strict positive rate for our Seurat/ESD-x intervention from 0.94 to 0.19 on the same images. In another failure case, bicycle interventions could produce a bicycle while omitting the prompted person.
To assess semantics directly, three human raters evaluated 80 cat/UCE images, with 20 images per condition. They judged whether the target was recognizable, whether the other specified entities and setting matched the prompt, and whether the specified scene relations held.
| Condition | Target recognizable | Other entities and setting match the prompt | Specified scene relations hold | Joint success |
|---|---|---|---|---|
| Original reference | 18/20 | 18/20 | 17/20 | 16/20 |
| Erased baseline | 1/20 | 17/20 | 16/20 | 1/20 |
| Matched random control | 2/20 | 17/20 | 16/20 | 1/20 |
| Intervention | 12/20 | 17/20 | 16/20 | 10/20 |
Each individual criterion uses a majority judgment. Joint success requires at least two of the three raters to each judge that the same image satisfies all three criteria. The intervention and matched random control had the same pass rates on the two individual scene criteria. In this cat/UCE sample, joint success was more frequent with the intervention (10/20 versus 1/20). We will describe the broader benchmark results as detector-defined positive rates.
Weakness 4. What the feature-level comparisons add
Prior work already demonstrates positive SAE steering in diffusion models [1] and restoration in unlearned classifiers [2]. Our audit measures paired target-versus-contrast responses, selects features based on their attenuation after erasure, and tests fixed directions derived from the discovery set. We agree that the original contrast-versus-rel comparison changes several components together.
To isolate the feature-selection choice in cat/UCE, we fixed the three hooks, SAE dictionaries, ten coordinates per hook, original-model mean-contrast prototype rule, and intervention strength (α = 32). We matched per-hook activation displacement norms after decoding and denormalization. The five selectors used the same 48 units as the fresh-scene evaluation, with separately fixed interventions.
| Feature selector | Strict positives at 0.25 | Mean target score |
|---|---|---|
| A(1−R) | 8/48 | 0.1125 |
| A | 1/48 | 0.0082 |
| AR | 0/48 | 0.0004 |
| Absolute mean contrast | 2/48 | 0.0329 |
| SAeUron-derived support | 21/48 | 0.3024 |
All matched random controls yielded 0/48 strict positives. SAeUron-derived coordinates performed best under the shared prototype rule and strength. This comparison identifies the stronger selector in this setting. The 8/48 result is from this controlled run, and the 7/48 in Part 1 is from the separately fixed fresh-scene run.
We also evaluated both models on identical latents, timesteps, and conditioning from the original model's denoising trajectory. Attenuation remained observable at these fixed inputs. The three top-ten feature sets shared only 2, 2, and 4 coordinates with those selected on the free trajectories (pooled Jaccard 0.154). The statistics used 80 discovery units. The activation cache also contained 75 older validation and test units. These results show that feature rankings depend on the trajectory even where attenuation persists at matched inputs. Downstream generation using directions selected under matched inputs remains untested.
References
[1] B. Cywiński and K. Deja. SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders. ICML, 2025.
[2] Y. Jang, J. Lee, D. Kim, J. Jo, and S. S. Woo. Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning. The ACM Web Conference (WWW), 2026.
Response to WdG1 (Part 3/3)
Weakness 5. Evaluation on a flow-based transformer
We ran a pilot on FLUX.1-schnell using Double Projections [3], with English springer as the target and cat as the guide concept. We trained three new SAEs on transformer activations and used 24 discovery units, 24 validation units, and 48 test units. The test templates and seeds were held out within six shared scene families. The hooks, prototype, and validation-selected strength (α = 32) were fixed before test generation.
| Condition | Mean target probability | Target probability ≥ 0.20 |
|---|---|---|
| Original reference | 0.2755 | 26/48 |
| Erased baseline | 0.0617 | 6/48 |
| Intervention | 0.1416 | 11/48 |
| Matched random control | 0.1070 | 8/48 |
The norm-matched random control also increased scores over the erased baseline. The paired mean gain of the intervention over this control was 0.0345, with a descriptive 95% bootstrap interval of [0.0128, 0.0662] after clustering by scene family. The pilot demonstrates that the audit can be applied to a flow-based transformer and estimates a modest additional classifier response for this concept and erasure method. We interpret these rates within the pilot because the classifier and threshold differ from those used for SD 1.4.
Question 1. Baseline and intervention paths in Figure 1
We will redraw Stage 5 as two labeled forward paths, “Erased baseline” and “Erased + intervention.” Both will start from the same erased weights, prompt, and initial noise, and both generated images will go to the same evaluator. A separate discovery panel will show how the dictionary, selected features, and prototype define the frozen direction, with an arrow feeding it into the intervention path at the selected denoising hooks.
Reference
[3] C. Zhang, J. Cheng, Z. Wang, and P. Liu. Closed-Form Concept Erasure via Double Projections. CVPR, 2026.
Response to WdG1 (Part 1/3)
Dear Reviewer WdG1,
Thank you for recognizing the evaluation gap, the fixed intervention design, and the use of controls across erasure methods and concept domains.
Weakness 1. What the audit establishes
SAE-Audit measures whether suppression under ordinary prompts withstands a fixed activation intervention at a specified strength. With the affine decoder and preserved residual, Eq. (10) reduces the intervention in Eq. (9) to a fixed additive displacement. The encoder contribution cancels at inference, and both the dictionary and the prototype supply information from the original model.
The dog-prompt control illustrates why this matters. Applying the pilot cat/ESD-x direction to dog prompts yielded cat detector positives in 29/40 images, compared with 3/40 for matched random support (Appendix K). This is consistent with synthesis aided by the injected information. Amplification and pathway bypass remain possible explanations for the responses we observe. The diagnostic result is the erased model's responsiveness to a specified direction under controlled conditions. The random and wrong-concept controls help establish which supplied directions are effective. We will state this scope alongside Eq. (2), including that a failed intervention constrains only the probe that was tested.
Weakness 2. Test-informed development and fresh-scene evaluation
We agree that the historical matrix is exploratory. Pilot test feedback informed the return to the mean prototype, the later rel comparison reused previously inspected scene families, and the scene-consistency rule was finalized after inspecting eleven audits. The contrast and rel configurations yielded 22/67 and 37/67 recovered cells, respectively. Table 1 reports 40/67 after selecting the stronger observed outcome for each cell. We will report both configuration-specific counts alongside the selected summary and describe the historical significance tests in light of these development choices.
The Kirchner eligibility floor of 0.0578 also used observed validation and test eligibility counts. The retained logs place that decision before the Kirchner intervention runs, when test outputs for other concepts were already available.
We ran a fresh cat/UCE evaluation using a public-weight reconstruction because the original private checkpoint and SAEs were unavailable. We fixed the probe, thresholds, and validation-selected strength (α = 32) before generating images for six new scene families. We retained all 48 units.
The paired mean detector-score gain over random was 0.1129, with a descriptive 95% bootstrap interval of [0.0469, 0.1949] after clustering by scene family. The strict positive rate increased by 14.6 percentage points, below the historical 20-point criterion for a recovered cell. This gives a response estimate on fresh scenes under a fixed protocol for one reconstructed setting.