NuST: Null-Space-Constrained Activation Steering for Robust Concept Erasure in Diffusion Models
Abstract
Text-to-image diffusion models can reproduce undesirable concepts inherited from large-scale web data, raising safety, copyright, and privacy concerns in real-world deployment. Existing concept-erasure methods typically fine-tune or edit model parameters, which can be computationally expensive and may inadvertently impair unrelated generation capabilities. Although activation steering provides a parameter-preserving alternative, unconstrained interventions can still perturb features shared by target and non-target concepts. We introduce NuST, a null-space-constrained activation-steering framework for robust concept erasure in diffusion models. At each selected module-timestep site, NuST constructs target-to-anchor shifts from paired prompts and learns a channel-wise transformation whose effective directions are restricted to the null space of retained benign activations. This design suppresses unintended changes to non-target representations while redirecting target activations toward safe anchor semantics. The transformations are obtained through closed-form regularized regression, requiring neither gradient-based optimization nor model-parameter updates. We evaluate NuST on nudity, artistic-style, and object erasure using Stable Diffusion v1.5. NuST reduces the average attack success rate for nudity recovery to 0.009% across five standard and adversarial benchmarks, achieves zero target-style classification accuracy for three artists, and lowers the average target-object success rate to 0.34%, while maintaining competitive generation quality and text-image alignment. These results establish null-space activation steering as an effective and robust alternative to parameter-based concept erasure.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.