Spectral Saliency-Guided Concept Erasure via Ordered Gradient Projection

Concept erasure aims to suppress a target concept in a generative model while preserving unrelated behavior. Existing methods commonly restrict either the coordinates or the directions of an update, although these operations need not commute. We propose Spectral Saliency-Guided Erasure (SSGE), which estimates low-rank forget and retain gradient subspaces once at initialization (the retain subspace is a denoising-gradient proxy), derives a continuous coordinate-wise gate from their loadings, and projects the gated forget gradient onto the orthogonal complement of the sampled retain subspace, without restricting updates to a single forget-derived adapter subspace. For an orthonormal retain basis, this ordering makes the resulting forget component orthogonal to the sampled basis; reversing the operator order or subsequently applying adaptive preconditioning need not preserve this property. On Stable Diffusion 1.5, SSGE reduces Van Gogh target CLIP from 0.3160 to 0.2279 across three training seeds, compared with 0.2678 for a protocol-matched deterministic UCE baseline, but produces larger changes on retain prompts. Across five Imagenette objects, suppression transfers to held-out synonyms and compositional prompts, while all adjacent-class prompt sets also lose alignment. SSGE therefore provides a well-defined local gradient constraint and consistent target suppression, but does not guarantee broad functional preservation.