acceptodds
Under review as a conference paper at ICLR 2027

What Remains After Concept Erasure? Sparse Autoencoder Probes for Auditing Diffusion Model

Abstract

Concept erasure aims to remove selected visual concepts from text-to-image diffusion models while preserving the generation of non-target concepts. Evaluation typically measures target suppression and non-target preservation in generated images, but a concept's absence from these outputs does not establish its removal from the model. We introduce SAE-Audit, a white-box audit using sparse autoencoder (SAE) probes to test whether activation interventions can restore target-concept generation without changing model weights or test prompts. After verifying target suppression, we compare the original and edited models using a shared SAE representation learned from original-model activations, identifying features whose target-related responses weaken after erasure. We inject sparse directions derived from the original-model responses into the edited model, keeping these directions fixed across test prompts, and evaluate recovery on held-out scene families. Experiments span six erasure methods and four domains: objects, artistic styles, copyrighted content, and nudity. Interventions using a few dozen selected SAE features yield higher average recovery rates than matched random-feature interventions, with recovery concentrated in objects and nudity following cross-attention edits. Most styles and copyrighted concepts show limited recovery under the tested intervention settings. These findings show that output suppression can coexist with responsiveness to sparse concept directions supplied from the original model.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine unlearning, AI safety, Concept Erasure, Sparse Autoencoder, Diffusion Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_WdG1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SAE-Audit, a white-box framework for testing concept erasure through fixed activation interventions. Sparse autoencoders identify concept-related responses that weaken after erasure, and directions derived from the original model are injected into the edited model without changing its weights or test prompts. Experiments cover six erasure methods and thirteen concepts on Stable Diffusion 1.4.

Critical strengths
  1. The paper addresses an important evaluation gap by testing whether concept suppression remains robust under targeted interventions on internal activations.

  2. The SAE-based framework connects changes in feature responses to recovery behavior while keeping model weights and test prompts unchanged.

  3. The experiments cover six erasure methods and four concept domains, with matched random controls and alternative recovery baselines that help assess the proposed interventions.

Critical weaknesses
  1. The injected directions and their values come from the original model and therefore carry target-concept information. Successful recovery could reflect bypassing a disrupted text-to-concept pathway, strengthening attenuated representations, or supplying information that the edited model no longer adequately contains. The current experiments do not distinguish these explanations. Although the paper appropriately acknowledges that recovery does not prove an intact representation survived, it remains unclear what the audit establishes about retained knowledge beyond responsiveness to externally supplied concept information. This limits its central diagnostic value.

  2. The later rel experiments reused scene families whose test results had already been inspected (Appendices C.3 and J). The pilot mean-versus-median prototype decision also used test feedback, and the scene-consistency rule was finalized after inspecting eleven audits (Appendix H.1). Table 1 further selects the stronger observed test outcome across two configurations. The disclosure is welcome, but these adaptive choices introduce possible selection bias. The results provide exploratory evidence, while the reported significance does not establish performance under an independently fixed evaluation protocol.

  3. Detector-level recovery does not reliably establish faithful recovery or scene preservation. Some interventions produce concept-associated textures or fragments while disrupting the requested scene. The Seurat relabeling experiment also shows substantial sensitivity to the detector’s label set. CLIP similarity alone cannot establish whether the target identity, other objects, and their relationships are preserved. A semantic assessment, potentially using a multimodal model calibrated against human judgments, would provide stronger evidence than the current similarity-based criterion.

  4. The additional diagnostic insight over existing recovery attacks remains limited. Prior work already demonstrates that concepts suppressed under ordinary prompts can be recovered. The contribution therefore depends on what feature-level probing reveals beyond this established observation. However, differences across concepts and hookpoints do not clearly separate the effects of the erasure method, the concept, and the probe itself. Changing the hookpoint criterion also changes the dictionaries, selected features, prototypes, and intervention strength. Consequently, the results do not isolate why recovery succeeds or fails. The modest and nonuniform advantage over Reverse-SAeUron further leaves the added value of the proposed feature-selection rule uncertain.

  5. Architectural generality is untested. All main experiments use the SD 1.4 UNet. It is unclear whether the feature-selection procedure, recovery behavior, and diagnostic conclusions extend to other backbones, such as SDXL or a DiT-based model.

Questions for authors

Could Figure 1 explicitly separate the baseline and intervention paths? Part 5 places both paths around the same central model illustration, with arrows pointing in opposite directions. This makes the generation and evaluation flow difficult to follow. Explicit labels for the two paths would clarify which computation is performed in each condition.

3 comments
AuthorsOpenReview ↗

Response to WdG1 (Part 1/3)

Dear Reviewer WdG1,

Thank you for recognizing the evaluation gap, the fixed intervention design, and the use of controls across erasure methods and concept domains.

Weakness 1. What the audit establishes

SAE-Audit measures whether suppression under ordinary prompts withstands a fixed activation intervention at a specified strength. With the affine decoder and preserved residual, Eq. (10) reduces the intervention in Eq. (9) to a fixed additive displacement. The encoder contribution cancels at inference, and both the dictionary and the prototype supply information from the original model.

The dog-prompt control illustrates why this matters. Applying the pilot cat/ESD-x direction to dog prompts yielded cat detector positives in 29/40 images, compared with 3/40 for matched random support (Appendix K). This is consistent with synthesis aided by the injected information. Amplification and pathway bypass remain possible explanations for the responses we observe. The diagnostic result is the erased model's responsiveness to a specified direction under controlled conditions. The random and wrong-concept controls help establish which supplied directions are effective. We will state this scope alongside Eq. (2), including that a failed intervention constrains only the probe that was tested.

Weakness 2. Test-informed development and fresh-scene evaluation

We agree that the historical matrix is exploratory. Pilot test feedback informed the return to the mean prototype, the later rel comparison reused previously inspected scene families, and the scene-consistency rule was finalized after inspecting eleven audits. The contrast and rel configurations yielded 22/67 and 37/67 recovered cells, respectively. Table 1 reports 40/67 after selecting the stronger observed outcome for each cell. We will report both configuration-specific counts alongside the selected summary and describe the historical significance tests in light of these development choices.

The Kirchner eligibility floor of 0.0578 also used observed validation and test eligibility counts. The retained logs place that decision before the Kirchner intervention runs, when test outputs for other concepts were already available.

We ran a fresh cat/UCE evaluation using a public-weight reconstruction because the original private checkpoint and SAEs were unavailable. We fixed the probe, thresholds, and validation-selected strength (α = 32) before generating images for six new scene families. We retained all 48 units.

ConditionStrict positives at 0.25
Original reference21/48
Erased baseline0/48
Intervention7/48
Matched random support0/48

The paired mean detector-score gain over random was 0.1129, with a descriptive 95% bootstrap interval of [0.0469, 0.1949] after clustering by scene family. The strict positive rate increased by 14.6 percentage points, below the historical 20-point criterion for a recovered cell. This gives a response estimate on fresh scenes under a fixed protocol for one reconstructed setting.

AuthorsOpenReview ↗

Response to WdG1 (Part 2/3)

Weakness 3. Target identity and scene fidelity

The Seurat relabeling result illustrates the limitations of detector scores. Adding pointillism as a competing label reduced the strict positive rate for our Seurat/ESD-x intervention from 0.94 to 0.19 on the same images. In another failure case, bicycle interventions could produce a bicycle while omitting the prompted person.

To assess semantics directly, three human raters evaluated 80 cat/UCE images, with 20 images per condition. They judged whether the target was recognizable, whether the other specified entities and setting matched the prompt, and whether the specified scene relations held.

ConditionTarget recognizableOther entities and setting match the promptSpecified scene relations holdJoint success
Original reference18/2018/2017/2016/20
Erased baseline1/2017/2016/201/20
Matched random control2/2017/2016/201/20
Intervention12/2017/2016/2010/20

Each individual criterion uses a majority judgment. Joint success requires at least two of the three raters to each judge that the same image satisfies all three criteria. The intervention and matched random control had the same pass rates on the two individual scene criteria. In this cat/UCE sample, joint success was more frequent with the intervention (10/20 versus 1/20). We will describe the broader benchmark results as detector-defined positive rates.

Weakness 4. What the feature-level comparisons add

Prior work already demonstrates positive SAE steering in diffusion models [1] and restoration in unlearned classifiers [2]. Our audit measures paired target-versus-contrast responses, selects features based on their attenuation after erasure, and tests fixed directions derived from the discovery set. We agree that the original contrast-versus-rel comparison changes several components together.

To isolate the feature-selection choice in cat/UCE, we fixed the three hooks, SAE dictionaries, ten coordinates per hook, original-model mean-contrast prototype rule, and intervention strength (α = 32). We matched per-hook activation displacement norms after decoding and denormalization. The five selectors used the same 48 units as the fresh-scene evaluation, with separately fixed interventions.

Feature selectorStrict positives at 0.25Mean target score
A(1−R)8/480.1125
A1/480.0082
AR0/480.0004
Absolute mean contrast2/480.0329
SAeUron-derived support21/480.3024

All matched random controls yielded 0/48 strict positives. SAeUron-derived coordinates performed best under the shared prototype rule and strength. This comparison identifies the stronger selector in this setting. The 8/48 result is from this controlled run, and the 7/48 in Part 1 is from the separately fixed fresh-scene run.

We also evaluated both models on identical latents, timesteps, and conditioning from the original model's denoising trajectory. Attenuation remained observable at these fixed inputs. The three top-ten feature sets shared only 2, 2, and 4 coordinates with those selected on the free trajectories (pooled Jaccard 0.154). The statistics used 80 discovery units. The activation cache also contained 75 older validation and test units. These results show that feature rankings depend on the trajectory even where attenuation persists at matched inputs. Downstream generation using directions selected under matched inputs remains untested.

References

[1] B. Cywiński and K. Deja. SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders. ICML, 2025.

[2] Y. Jang, J. Lee, D. Kim, J. Jo, and S. S. Woo. Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning. The ACM Web Conference (WWW), 2026.

AuthorsOpenReview ↗

Response to WdG1 (Part 3/3)

Weakness 5. Evaluation on a flow-based transformer

We ran a pilot on FLUX.1-schnell using Double Projections [3], with English springer as the target and cat as the guide concept. We trained three new SAEs on transformer activations and used 24 discovery units, 24 validation units, and 48 test units. The test templates and seeds were held out within six shared scene families. The hooks, prototype, and validation-selected strength (α = 32) were fixed before test generation.

ConditionMean target probabilityTarget probability ≥ 0.20
Original reference0.275526/48
Erased baseline0.06176/48
Intervention0.141611/48
Matched random control0.10708/48

The norm-matched random control also increased scores over the erased baseline. The paired mean gain of the intervention over this control was 0.0345, with a descriptive 95% bootstrap interval of [0.0128, 0.0662] after clustering by scene family. The pilot demonstrates that the audit can be applied to a flow-based transformer and estimates a modest additional classifier response for this concept and erasure method. We interpret these rates within the pilot because the classifier and threshold differ from those used for SD 1.4.

Question 1. Baseline and intervention paths in Figure 1

We will redraw Stage 5 as two labeled forward paths, “Erased baseline” and “Erased + intervention.” Both will start from the same erased weights, prompt, and initial noise, and both generated images will go to the same evaluator. A separate discovery panel will show how the dictionary, selected features, and prototype define the frozen direction, with an arrow feeding it into the intervention path at the selected denoising hooks.

Reference

[3] C. Zhang, J. Cheng, Z. Wang, and P. Liu. Closed-Form Concept Erasure via Double Projections. CVPR, 2026.

Reviewer_D66wOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces an approach to audit concept erasure in diffusion models using Sparse Autoencoders (SAE). The method trains SAEs on activations from the original model where feature are identified whose concept responses weaken after erasure has been applied. It then uses the original model to construct an activation intervention which stays fixed across evaluation and does not alter model weights. The experimental results cover 6 erasure methods and 13 concepts on Stable Diffusion 1.4. The results show strong recovery for objects and nudity. These results show that erased models can recover concepts.

Critical strengths
  1. The evaluation is comprehensive as it includes a two step process where the erasure is verified first then tests the intervention on a held out set.
  2. The intervention is fixed and does not optimize for each new prompt. The evaluation is performed using the same iintervention.
  3. The comparisons with baselines help measure the benefit of the feature-selection method.
  4. The paper reports interesting failure cases and shows that high detector scores can come textures or incomplete objects.
Critical weaknesses
  1. Detector scores do not reliably prove concept recovery because some outputs can contain fragments, textures, incomplete scenes. The scene similarity metric does not verify concept identity. This needs a human evaluation.
  2. The improvements over Reverse-SAeUron is not high as the mean recovery rate improves from 0.35 to 0.40 under one layer-selection rule and 0.19 to 0.21 under the other rule. Moreover reverse SAeUron performs better on copyrighted concepts.
  3. The evaluation is limited to Stable Diffusion 1.4. There are many newer and state of the art models such as SD 3, FLUX on which the same concept erasure baselines have been tested.
Questions for authors
  1. The original and erased models follow different denoising trajectories during activation collection. How does the analysis separate the effect of changed model weights from the effect of different intermediate inputs?
  2. How does SAE-Audit differ from SAE-based restoration framework of [1]?

[1] Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning.

3 comments
AuthorsOpenReview ↗

Response to D66w (Part 1/3)

Dear Reviewer D66w,

Thank you for recognizing the two-stage evaluation, the fixed intervention across prompts, and the analysis of failure cases.

Weakness 1. Semantic recovery and human evaluation

We agree that human evaluation is needed to establish whether the target is recognizable and the image meets the specified scene requirements. Appendix C.3 illustrates the limits of detector scores. Adding pointillism as a competing label reduces the strict positive rate for our Seurat/ESD-x intervention from 0.94 to 0.19 on the same images.

To address this concern, three human raters evaluated 80 cat/UCE images, with 20 images per condition. They judged whether the target was recognizable, whether the other specified entities and setting matched the prompt, and whether the specified scene relations held.

ConditionTarget recognizableOther entities and setting match the promptSpecified scene relations holdJoint success
Original reference18/2018/2017/2016/20
Erased baseline1/2017/2016/201/20
Matched random control2/2017/2016/201/20
Intervention12/2017/2016/2010/20

Each individual criterion uses a majority judgment. An image meets the joint criterion when at least two of the three raters each judge that it satisfies all three criteria.

The joint criterion was met in 10/20 intervention images and 1/20 matched random-control images, a difference of 45 percentage points in this cat/UCE sample. The two conditions had the same pass rates on the individual scene criteria. The human evaluation therefore shows that more intervention images contained a recognizable target while meeting the same prompt-specified scene criteria. We will report the broader benchmark results as detector-defined positive rates.

AuthorsOpenReview ↗

Response to D66w (Part 2/3)

Weakness 2. Comparison with Reverse-SAeUron

The aggregate gains are modest. Across the 67 audited cells, the mean strict positive rate is 0.40 for our method versus 0.35 for adapted Reverse-SAeUron at rel-selected hooks, and 0.21 versus 0.19 at contrast-selected hooks. At rel-selected hooks, adapted Reverse-SAeUron leads on copyrighted concepts (0.20 versus 0.17) and has a higher strict positive rate in 23 of the 67 individual comparisons. We will describe these comparisons as exploratory because prototype development used pilot test feedback and the rel setting reused previously inspected scene families.

We also compared five feature selectors on 48 fresh cat/UCE units using a public-weight reconstruction. We fixed the three hooks, SAE dictionaries, ten coordinates per hook, prototype construction rule, sampling settings, and intervention strength (α = 32, selected on validation for the parent circuit). We matched the displacement norm at each hook in the model's activation space after decoding and denormalization.

Feature selectorStrict positives at 0.25Mean target score
A(1−R)8/480.1125
A1/480.0082
AR0/480.0004
Absolute mean contrast2/480.0329
SAeUron-derived support21/480.3024

Each matched random control yielded 0/48 strict positives. SAeUron-derived coordinates performed best at the shared strength. That condition uses our prototype construction rule on coordinates selected by SAeUron. The result therefore compares feature selectors under a common intervention procedure. Tables 2 and 17 report the separate comparison with the adapted Reverse-SAeUron baseline.

Question 2. Relationship to prior SAE-based restoration

Jang et al. [1] select class-discriminative SAE features by class-wise F1 in a vision transformer and use the original model's activations for each input image to steer the selected features of the unlearned model. SAeUron [2] also demonstrates positive steering of style features using concept-average SAE activations in Appendix H.

In SAE-Audit, we measure paired target-versus-contrast responses in a shared, frozen SAE basis and select features whose responses weaken after erasure. For each audit, we construct one direction from discovery-set activations and keep it fixed across test prompts and seeds. Both the dictionary and the direction carry information from the original model. The experiment measures how the erased model responds to that direction, leaving open how much of the information behind the response was retained by the erased model.

References

[1] Y. Jang, J. Lee, D. Kim, J. Jo, and S. S. Woo. Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearning. The ACM Web Conference (WWW), 2026.

[2] B. Cywiński and K. Deja. SAeUron: Interpretable Concept Unlearning in Diffusion Models with Sparse Autoencoders. ICML, 2025.

AuthorsOpenReview ↗

Response to D66w (Part 3/3)

Weakness 3. Evaluation on FLUX

We ran a pilot on FLUX.1-schnell with public weights and Double Projections [3], using English springer as the target and cat as the guide concept. We used 24 discovery units, 24 validation units, and 48 test units with held-out templates and seeds within six scene families. We trained three new SAEs and fixed the hooks, prototype, and validation-selected strength (α = 32) before generating the test images.

ConditionMean target probabilityTarget probability ≥ 0.20
Original reference0.275526/48
Erased baseline0.06176/48
Intervention0.141611/48
Matched random control0.10708/48

The random control matched the displacement norm at each hook and also increased scores over the erased baseline. The paired mean gain of our intervention over this control was 0.0345, with a descriptive 95% bootstrap interval of [0.0128, 0.0662] after clustering by scene family. The pilot shows that the audit can be applied to a flow-based transformer and finds a modest additional classifier response for one concept and one erasure method. The classifier and threshold differ from those used for SD 1.4, so we interpret the rates within this pilot.

Question 1. Separating weight changes from intermediate inputs

In the main analysis, each model follows its own denoising trajectory. Matching the initial noise still allows the intermediate latents to diverge. In a separate cat/UCE diagnostic, we evaluated both models on the same latents, timesteps, and conditioning from the original model's trajectory, using the same three hooks and frozen SAEs. This holds the inputs fixed, so the measured activation differences at those states reflect the model changes.

Attenuation persisted under matched inputs. The top-ten feature sets from the two analyses shared 2, 2, and 4 features across the three hooks, with a pooled Jaccard index of 0.154 and rank correlations of 0.276–0.310. Among features selected on the free trajectories, the median retained-response statistic R under matched inputs was 0.1023, 0.0320, and 0.1345 at the three hooks, respectively.

Feature statistics, feature selection, and prototype construction used only the 80 discovery rows. The activation cache also held 37 old validation and 38 old test units. Downstream generation with directions selected under matched inputs remains untested. We will report the matched-input result as evidence of attenuation at the replayed states and describe the main feature rankings as trajectory-dependent.

Reference

[3] C. Zhang, J. Cheng, Z. Wang, and P. Liu. Closed-Form Concept Erasure via Double Projections. CVPR, 2026.

Reviewer_TySfOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

SAE-Audit is a white-box audit that uses sparse autoencoder (SAE) probes to test whether activation interventions can restore target-concept generation without changing model weights or test prompts. The experiments evaluate the erasure of different types of concepts, including four objects, four artistic styles, four copyrighted characters and logos, and one NSFW concept.

Critical strengths
  1. The probing strategy is an interesting idea that sheds light on the persistence of residual information about the target concept even after erasure.

  2. The five-stage SAE-Audit pipeline is well designed for constructing these probes and evaluating target-concept recovery.

  3. The experimental section is also thorough.

Critical weaknesses

Manuscript Formatting, Presentation, and Methodology:

The paper is difficult to follow. The writing is hard to understand, and the terminology is confusing.

For instance:

  1. The paper uses nonstandard terminology such as “weight editing,” which I have not encountered in the prior literature.

  2. Which two edits are the authors referring to in the phrase “two edits” on line 47? Do other approaches also fall under the category of editing? The authors appear to conflate editing and erasure, although these are distinct tasks.

  3. In the phrase “concept-related activations differently” on line 48, are the authors referring to activations associated with the target concept or with related concepts?

  4. In the phrase “identify concept-related responses” on line 61, where are these responses identified?

The audit objective in Equation 2 is insufficiently validated. Its dependence on the intervention also raises concerns about its generalizability across different concepts.

The symbol τ is undefined in Equation 3. Additionally, the eligibility criteria are not clearly defined, and the current wording is difficult to comprehend. The authors should explain what these criteria do to improve readability.

What are the “discovery scores” mentioned on line 189?

Experimental Evaluation:

The baselines used for comparison in Table 1 are outdated; SDErasure is the only baseline from 2026.

The evaluation involving only four object classes is not sufficiently extensive. Prior studies use standard datasets such as ImageNet or COCO. The authors should select object classes from these standard benchmarks and evaluate their method across more classes.

The method is tailored to the U-Net architecture of Stable Diffusion v1.4, which is outdated. Experiments on recent flow-based models, such as FLUX, are missing.

Furthermore, architectures such as FLUX do not rely on the same fixed encoder–decoder structure as U-Net. Consequently, the proposed probing approach may not work as intended across these newer architectures.

Questions for authors

Please see the weaknesses section.

3 comments
AuthorsOpenReview ↗

Response to TySf (Part 1/3)

Dear Reviewer TySf,

Thank you for recognizing the probing idea, the five-stage pipeline, and the scope of the evaluation. We address your questions about the audit objective, threshold calibration, and experimental coverage below.

Weakness 1. Terminology, response location, and the audit objective

The cross-concept evidence for Eq. (2) comes from 67 eligible concept–eraser pairs on SD v1.4, drawn from 13 concepts and six erasure methods. Reported separately, the contrast and relative hookpoint rankings met the detector-based recovery criterion in 22/67 and 37/67 pairs, respectively. Recovery was more common for objects than for styles or copyrighted concepts. Table 1 displays the stronger observed result for each pair. We treat these results as exploratory because the prototype choice used pilot test feedback and the later relative-ranking comparison reused test scenes after the contrast results had been inspected. In the two fresh evaluations, cat/UCE and English springer/FLUX, fixed interventions raised mean detector scores above the corresponding random controls. These results establish responsiveness in the two tested settings. Broader cross-concept validation remains open.

We will clarify the introduction and the audit definition as follows.

  1. Erasure and implementation. Concept erasure aims to suppress a target while preserving non-target generation. We will use “parameter-updating concept-erasure methods” for baselines that change model weights. By “two edits” on line 47, we mean any two concept-erasure procedures. We will revise the sentence to read “Two concept-erasure procedures can suppress the target to a similar degree at the output while affecting target-associated activations differently.”
  2. Target-associated responses. “Concept-related” refers to the target itself. We estimate its association with an activation by comparing prompts that describe the same scene and replace the target with a contrast concept, such as cat versus dog. We measure the responses at selected intermediate blocks in the conditional branch during denoising. We train the SAEs on discovery activations from the original model and freeze them, so both models are compared in the same feature coordinates.
  3. Equation (2). For the same erased model, prompt, and initial noise, Eq. (2) measures the paired change in evaluator score between applying a specified intervention and leaving the activations unmodified. The random intervention is a separate control for whether the selected directions matter. The dictionary and prototype both carry information from the original model, so the effect depends on the supplied probe. Whether the target is recognizable and the scene is preserved requires separate assessment.

For the fresh cat/UCE test, we reconstructed the setup using public weights because the original private checkpoints were unavailable. We retained all 48 units from six new scene families. At the strict cutoff of 0.25, our intervention yielded 7/48 positives, the erased model and random control each yielded 0/48, and the original model yielded 21/48. The paired mean-score gain over random was 0.1129, with a descriptive 95% interval of [0.0469, 0.1949] after clustering by scene family. The 14.6-percentage-point gain in strict positives was below the 20-point margin required by the original recovery criterion.

In a matched-input cat/UCE diagnostic, attenuation remained observable when we held the latent, timestep, and conditioning fixed, but the feature rankings changed. The pooled Jaccard similarity between the top-10 sets was 0.154. The statistics used 80 discovery units from a cache of 155 units that also contained earlier validation and test activations. Downstream generation with the directions from this diagnostic remains unevaluated.

In a separate selector comparison on the same 48 fresh units, the SAeUron-derived feature set yielded 21/48 strict positives and our attenuation-based set yielded 8/48. Both used the same hooks, SAE dictionaries, prototype rule, intervention strength, and matched displacement norms. The 8/48 result comes from this controlled selector comparison. The 7/48 result comes from the fresh-scene test above.

AuthorsOpenReview ↗

Response to TySf (Part 2/3)

Weakness 2. Thresholds, eligibility, and discovery scores

Appendix A lists the thresholds, and we agree that their roles and calibration should be explained beside Eq. (3). A prompt pair and seed are eligible if all three conditions hold.

  1. Target elicitation. The original model's image from the target prompt must score at least τc+\tau_c^+. This checks that the evaluator detects the target when prompted.
  2. Target leakage. The original model's image from the matched contrast prompt must score at most τc−\tau_c^-. This screens for target leakage in the contrast condition.
  3. Suppression before intervention. The erased model's image from the target prompt must score at most τcE\tau_c^E. This checks that the target score is low before we apply the probe.

If the first two checks pass and the third fails, we record a direct erasure failure. A concept–eraser pair in Table 1 requires at least 80 eligible units overall and at least 16 each in validation and test. The strict cutoff for measuring recovery after intervention is a separate threshold. The new cat/UCE and FLUX experiments each retained all 48 fresh units without this eligibility filter.

The “discovery scores” on line 189 are the target evaluator's scores on images generated by the original model from the target and contrast prompts in the discovery split. The implementation selects τc+\tau_c^+ to maximize recall at a precision of at least 0.95, with the 25th percentile of target-prompt scores as the fallback. It uses the 90th percentile of contrast-prompt scores for τc−\tau_c^- and the midpoint of the two thresholds for τcE\tau_c^E.

We will revise the manuscript's description of discovery-only calibration to disclose the Kirchner exception. The retained logs place the 0.0578 floor before the Kirchner intervention runs, but the adjustment used observed validation and test eligibility counts. Test outputs for other concepts were already available. The original prototype choice and the later relative-ranking comparison also used test feedback. For the new runs, we fixed the intervention sites, prototypes, and validation-selected strength before evaluating on fresh inputs.

AuthorsOpenReview ↗

Response to TySf (Part 3/3)

Weakness 3. Recent erasure methods, object coverage, and FLUX

  1. Recent erasure methods. Table 1 includes SPEED, published at ICLR 2026 [1], alongside SDErasure. The additional FLUX experiment uses Double Projections, published at CVPR 2026 [2].
  2. Object coverage. The original evaluation uses four COCO categories: cat, bicycle, truck, and fire hydrant. The FLUX experiment adds English springer, an ImageNet category. These categories are evaluated on images generated from our prompt sets. We agree that five object categories still provide limited coverage.
  3. Compatibility with FLUX. We trained new SAEs on image-token activations from FLUX.1-schnell and applied the intervention at selected transformer blocks. The SAE encoder maps the image-token activations into sparse feature coordinates. The SAE decoder maps the selected feature direction back into the transformer's activation space.

For English springer, we used 24 discovery units and 24 validation units, then evaluated on 48 fresh units with held-out prompt templates and seeds within the same six scene families. Validation selected the intervention strength (α=32\alpha = 32) before evaluation. We retained all 48 units. The random intervention matched the raw activation displacement norm at each hook.

ConditionMean target probabilityUnits with probability ≥ 0.2
Original model0.275526/48
Erased model without intervention0.06176/48
Erased model with our intervention0.141611/48
Erased model with norm-matched random intervention0.10708/48

The random intervention also raised the target score. The paired mean gain of our intervention over random was 0.0345, with a descriptive 95% interval of [0.0128, 0.0662] after clustering by scene family. The experiment shows that the audit can run on a flow-based transformer and measures a modest additional detector response for the tested checkpoint, erasure method, and object class.

References

[1] O. Li, Y. Wang, X. Hu, H. Jiang, J. Tao, Y. Hao, J. Ma, and F. Feng. “SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Models.” ICLR, 2026.

[2] C. Zhang, J. Cheng, Z. Wang, and P. Liu. “Closed-Form Concept Erasure via Double Projections.” CVPR, 2026, pp. 24503–24513.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.