acceptodds
Withdrawn from ICLR 2027

NuST: Null-Space-Constrained Activation Steering for Robust Concept Erasure in Diffusion Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Text-to-image diffusion models can reproduce undesirable concepts inherited from large-scale web data, raising safety, copyright, and privacy concerns in real-world deployment. Existing concept-erasure methods typically fine-tune or edit model parameters, which can be computationally expensive and may inadvertently impair unrelated generation capabilities. Although activation steering provides a parameter-preserving alternative, unconstrained interventions can still perturb features shared by target and non-target concepts. We introduce NuST, a null-space-constrained activation-steering framework for robust concept erasure in diffusion models. At each selected module-timestep site, NuST constructs target-to-anchor shifts from paired prompts and learns a channel-wise transformation whose effective directions are restricted to the null space of retained benign activations. This design suppresses unintended changes to non-target representations while redirecting target activations toward safe anchor semantics. The transformations are obtained through closed-form regularized regression, requiring neither gradient-based optimization nor model-parameter updates. We evaluate NuST on nudity, artistic-style, and object erasure using Stable Diffusion v1.5. NuST reduces the average attack success rate for nudity recovery to 0.009% across five standard and adversarial benchmarks, achieves zero target-style classification accuracy for three artists, and lowers the average target-object success rate to 0.34%, while maintaining competitive generation quality and text-image alignment. These results establish null-space activation steering as an effective and robust alternative to parameter-based concept erasure.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Concept Erasure, Activation Steering, Null-Space Constraint, Text-to-Image Diffusion Models, Generative Model Safety

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_aw9tOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NuST, a concept erasure method that needs no update to the diffusion model parameters. It collects activations from feed-forward modules and denoising timesteps, builds a null space from the spatial mean of benign prompts, and then learns a target-to-anchor transform by closed-form regularized regression. At inference, the transform is applied position by position, which suppresses the target concept while keeping other generation abilities intact. The authors test nudity, artistic style, and object erasure on Stable Diffusion 1.5. They evaluate robustness and generation quality with prompt attacks, classification accuracy, FID, and CLIP metrics, and they report a low target recovery rate.

Critical strengths
  1. The idea is clear. It steers the target concept toward a safe anchor, while it constrains the disturbance to benign representations.
  2. The closed-form solution needs no backpropagation and changes no original model parameter, so deployment stays simple.
  3. The experiments cover three erasure tasks, and they report target suppression together with non-target preservation.
Critical weaknesses
  1. The key baseline numbers are inconsistent. In Table 1, ActErase detects only 6 exposed body sites, yet Table 2 reports a 1.9% success rate on I2P images. With the 4703 images stated in the paper and one shared detection protocol, the two cannot correspond.
  2. The preservation guarantee is narrow. The null space constraint protects only the spatial mean activation, but inference applies the transform position by position. That does not directly guarantee that the actual benign representations stay unchanged.
  3. The core mechanism lacks a direct ablation. There is no control that holds every other setting fixed and removes only the null space constraint, so its independent contribution is hard to confirm.
  4. The loss in non-target capability is clear. Mean non-target object accuracy falls from 81.36% to 69.41%, and after erasing Church it is only 54.15%. That is hard to square with the word non-destructive.
  5. The practical cost is not light. Generating 1000 images takes 6313 seconds, about 7.9 times that of ActErase, and it needs another 6,135 seconds of preparation. (Table 6)
Questions for authors
  1. Do Table 1 and Table 2 use the same I2P samples, the same detection categories, and the same denominator?
  2. How much do the erasure rate and the preservation rate change once the null space constraint is removed?
  3. If the spatial mean is unchanged, how do you guarantee that a position-wise intervention does not damage benign content?
  4. What are the eigenvalue threshold and the lower bound on the dimension for the adaptive null space?
  5. Were the adversarial prompts re-optimized against the full model that includes the NuST intervention?
  6. Are the results stable when the retained prompts or the random seed change?
  7. With another sampler or with 20 sampling steps, do all the transforms have to be rebuilt?
  8. The original model already has zero classification accuracy on Da Vinci. How does that test show whether erasure works?
Reviewer_1V9LOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NuST, a training-free concept erasure framework for text-to-image diffusion models based on null-space-constrained activation steering. The method aims to suppress undesirable concepts while preserving the generation capabilities of non-target concepts. Specifically, NuST constructs a null-space projector from benign activations and learns site-specific steering transformations through closed-form regularized regression using paired target and anchor prompts. During inference, the learned transformations are applied to intermediate FFN activations at selected denoising timesteps without modifying the pretrained model parameters. Experiments on Stable Diffusion v1.5 cover nudity, artistic-style, and object erasure, demonstrating strong erasure effectiveness and adversarial robustness compared with existing methods.

Critical strengths
  1. Clear motivation and reasonable methodology. The paper addresses an important challenge in concept erasure: suppressing target concepts while minimizing unintended changes to non-target representations. The use of null-space constraints provides a reasonable formulation for protecting benign activations during steering.

  2. Strong erasure effectiveness and adversarial robustness. NuST achieves competitive results across nudity, artistic-style, and object erasure tasks. In particular, its performance against multiple adversarial attacks demonstrates the effectiveness of activation-space intervention for suppressing undesirable concepts.

  3. Training-free design with a closed-form solution. The proposed method does not require gradient-based optimization or modifications to the pretrained model parameters. The steering transformations can be computed offline and reused during inference, providing flexibility for concept-specific intervention.

Critical weaknesses
  1. Limited advantages in non-target concept preservation. Preserving non-target concepts is a central motivation of this work, yet the experimental results do not demonstrate a clear advantage. In Table 2, NuST achieves only moderate FID and CLIP scores compared with existing methods. Similarly, Table 3 does not establish a meaningful preservation advantage, as single-concept style erasure generally has a limited impact on overall generation capabilities. In Table 4, NuST also underperforms several baselines in non-target accuracy. Moreover, I would appreciate further clarification on why some baseline methods exhibit highly inconsistent performance across object categories, given that these methods are generally expected to perform reasonably well in single-concept erasure. Could this be attributed to suboptimal hyperparameter configurations? The authors should clarify the baseline configurations and ensure that all methods are evaluated under appropriately tuned settings.

  2. Insufficient ablation on the core null-space constraint. The authors should compare the proposed null-space constraint with simpler alternatives, such as jointly optimizing erasure and preservation objectives through weighted regression, similar to UCE. Such a comparison would help determine whether the null-space formulation offers meaningful advantages over conventional approaches that explicitly balance erasure and preservation. The current experiments do not sufficiently establish the necessity of the proposed constraint.

  3. Limited technical novelty and insufficient distinction from AlphaSteer. NuST is closely related to AlphaSteer, which has already introduced null-space-constrained activation steering. The main methodological difference appears to be its application to diffusion U-Net FFN activations with timestep-specific transformations. However, the paper does not adequately explain why directly applying AlphaSteer to diffusion models would be insufficient, what diffusion-specific challenges arise from such an adaptation, or which proposed designs specifically address these challenges. A more detailed methodological comparison and corresponding experimental evidence are needed to establish the novelty and necessity of NuST beyond transferring an existing framework to a different model architecture.

  4. Significant computational overhead and unclear practical significance. Despite being training-free, NuST incurs substantial computational costs. According to Table 6, its total runtime is 12,448 seconds, compared with approximately 1,150 seconds for AdaVD, with image generation alone being about 5.5 times slower. More fundamentally, I question the practical necessity of training-free approaches for concept erasure. Since NuST relies on inference-time interventions without modifying model parameters, an adversary with sufficient white-box access could potentially disable or bypass the steering mechanism. This raises concerns about its applicability in real-world safety-critical scenarios. In particular, the authors should clarify the threat model underlying their evaluations against attacks such as P4D and UnlearnDiff (UD), and explain what practical security guarantees NuST can provide when the attacker has access to the underlying model and its inference pipeline.

  5. Limited generalization across model architectures. All experiments are conducted on Stable Diffusion v1.5. Since NuST relies on architecture-specific FFN activations and timestep-dependent transformations, its effectiveness on newer models such as SDXL and FLUX remains unclear.

  6. Insufficient support for some experimental claims. In artistic-style erasure, the original SD v1.5 already achieves 0% classification accuracy for Da Vinci, making the reported erasure result inconclusive. Additionally, the object erasure results reveal a considerable erasure–preservation trade-off that deserves further analysis.

  7. Concerns about the reliability of adversarial evaluation results. I have some concerns regarding the UnlearnDiff (UD) results in Table 2. UD is a well-established white-box attack, and the original STEREO paper reports an ASR of 4.21% under this attack, whereas this paper reports 0.00% for STEREO. Based on my experience, achieving a zero ASR against UD is difficult. Although differences in experimental settings may explain this discrepancy, I strongly encourage the authors to verify their evaluation using the official UD implementation and clarify the attack configurations to ensure a fair and reliable comparison.

Questions for authors

See the weakness part.

Reviewer_QoVLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes NuST, a concept-erasure method that applies null-space-constrained linear transformations to diffusion-model FFN activations without modifying pretrained weights. At each selected module–timestep pair, it fits target–anchor activation shifts while constraining changes to retained benign representations. Experiments on SD v1.5 report strong suppression across nudity, artistic styles, and objects.

Critical strengths
  1. The reported experiments show target suppression across nudity, artistic styles, and object categories, including the evaluated adversarial prompts. This provides evidence that the approach works across different types of concepts within the tested model, rather than only for a single narrowly defined target.

  2. NuST constructs steering operators through closed-form regression without updating pretrained weights. This offers a way to apply concept suppression while retaining the original model parameters, although the additional inference cost remains a separate practical limitation.

Critical weaknesses
  1. The contribution beyond AlphaSteer is not sufficiently established. Both methods combine benign-activation null-space constraints with regularized regression for input-dependent steering. NuST introduces diffusion-specific adaptations, but the existing ablations do not clearly establish how much the central null-space constraint and timestep-specific fitting contribute beyond transferring the existing framework to a new setting.

  2. The reported generation time is substantially longer than that of the compared methods. Unlike preparation or fine-tuning, this overhead recurs during deployment. The training-free formulation therefore does not establish practical efficiency, and the paper provides limited explanation of whether the slowdown is inherent to the method or attributable to its implementation.

Questions for authors
  1. What is the main contribution beyond AlphaSteer, and what evidence distinguishes the benefits of NuST’s design from those of adapting the existing framework?

  2. SPEED and CASteer are discussed in related work but omitted from quantitative comparisons. Their relevance to null-space preservation and inference-time steering makes them important alternatives. Without these comparisons, it remains unclear whether NuST offers a better balance of erasure, preservation, and computational cost. How does NuST compare with SPEED and CASteer when erasure effectiveness, non-target preservation, and computational cost are considered together?

  3. All experiments use SD v1.5, leaving architectural generalization untested. Although newer backbones are mentioned as future work, the paper does not establish whether its intervention strategy remains effective in DiT-based T2I models such as FLUX and PixArt. Given the field’s shift toward these architectures, restricting evaluation to SD v1.5 also weakens the timeliness and practical relevance of the contribution, even if the results remain valid for the evaluated backbone. How would the method transfer to DiT-based T2I models (e.g. FLUX, PixArt), and is there evidence that its effectiveness extends beyond SD v1.5?

  4. The qualitative examples and aggregate metrics do not sufficiently distinguish selective erasure from broader output degradation. The paper acknowledges distortions and over-suppression, while evaluation on general COCO prompts does not directly establish preservation of surrounding content in target-containing prompts. The quantitative comparisons also show no consistent advantage in non-target preservation or generation quality, leaving it unclear whether the stronger target suppression translates into a better overall erasure–preservation trade-off. Could you provide further evidence that the method selectively removes target concepts while preserving non-target content?

  5. Although the appendix describes prompt construction, it does not provide enough information to reconstruct the exact prompt sets, retained data, and adaptive null-space selection. This matters because the paper’s own ablation shows sensitivity to prompt-set construction. The missing details limit reproducibility and assessment of how reliably the results generalize beyond the chosen data. Could you provide the missing data-construction details and null-space selection criteria?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.