A Concept Is More Than the Sum of Its Parts: Concept Erasure by Excising the Surplus

Concept erasure aims to remove undesired concepts from text-to-image diffusion models without harming their ability to generate other concepts. Recent training-free methods tackle the problem by suppressing a direction that represents the target concept. However, this direction also carries parts the concept shares with similar ones (e.g., the wheels a garbage truck shares with cars), damaging these nearby concepts as well. Prior works mitigate this issue by protecting a retain set of related concepts, yet their performance is sensitive to which neighbors are chosen. Instead of searching for better neighbors outside the concept, we look inside the concept itself and let the parts of the concept decide what to keep. Since the same parts can form different wholes, a concept is more than the sum of its parts. We call this extra component the concept surplus: the portion of its features that its parts cannot explain. Thus, erasing only this surplus prevents the model from assembling the concept. Meanwhile, its parts remain intact, and so does every concept that shares them. Based on this motivation, we propose to **Ex**tract the **C**oncept **I**dentity as its **S**urplus over its parts, and **E**rase it. With the same principle, our method, named **ExCISE**, can erase a single part or a subset thereof, such as nudity, while preserving the rest of the person.