acceptodds
Withdrawn from ICLR 2027

Erased But Not Gone: How Model Merging Weakens Concept Erasure in Diffusion Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Concept erasure suppresses a target concept in a diffusion model, but the edited weights may later be merged with another checkpoint. We study whether suppression survives composition with a public style or domain model. Averaging weakens the erasure update as it incorporates the partner, whereas additive task arithmetic keeps that update intact. We probe these changes separately and compare merge operators across suppression and partner-appearance transfer. Our study covers three object concepts in Stable Diffusion 1.5, with a 95-setting checkpoint/partner grid and a balanced 33-setting strength sweep. Attenuation restores target-concept evidence, but partner addition can also restore it without weakening the erasure update. At common minimum appearance-transfer targets, addition improves normalized suppression retention by 26.8 percentage points over interpolation; the tested parameter guards add 1.0 point over addition. These paired comparisons are retrospective and depend on the measured appearance estimator. The results identify strength-tuned addition as an important baseline and show why preserving an update in parameter space does not guarantee preservation of its behavioral effect.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
diffusion models, concept erasure, machine unlearning, model merging, task arithmetic, model editing, safety alignment

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_FmqZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the robustness of concept erasure in diffusion models under model merging. The authors distinguish between erasure-update attenuation and partner-update injection, showing that both can restore erased concepts. Through experiments on Stable Diffusion 1.5, the paper compares interpolation, task arithmetic, parameter guards, and re-erasure. The results suggest that strength-tuned addition provides a better suppression–appearance trade-off than interpolation, while parameter guards offer only marginal additional improvements.

Critical strengths
  1. The research problem is meaningful. The paper explores an important but relatively underexplored scenario in which concept erasure may be compromised by subsequent model merging, even without adversarial intent.

  2. The experimental controls is well-designed. The separation of erasure attenuation and partner injection, together with norm-matched random perturbations, provides useful empirical evidence for understanding concept recovery.

  3. The comparison of merging strategies is well-designed. The appearance-constrained evaluation accounts for the trade-off between erasure preservation and appearance transfer, offering practical insights into different merging operators.

Critical weaknesses
  1. The paper systematically evaluates concept resurgence under different model merging strategies. However, the underlying mechanisms remain insufficiently explored. For example, while parameter guards improve erasure retention, they often substantially compromise the transfer of partner capabilities. The paper provides limited explanation for this trade-off or insight into how erasure-related and partner-related parameter updates interact. A deeper analysis would strengthen the paper's contributions beyond empirical observations.

  2. The experiments are restricted to three benign object concepts, leaving generalization to other kinds of concepts unclear. Additionally, erasure effectiveness relies primarily on classifier scores, while appearance transfer is measured using CLIP similarity, which mixes style, content, and composition. These limitations weaken the reliability and generality of the conclusions.

Questions for authors
  1. Could the authors provide further insights into the mechanisms by which partner updates restore erased concepts, beyond the observed differences between structured updates and random perturbations?

  2. To what extent do the findings generalize to other concept categories, particularly identities, artistic styles, and harmful content? In particular, it would be interesting to investigate whether artistic style erasure remains effective when the erased model is merged with other style-specific models.

Reviewer_NDwGOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether concept erasure survives merging a diffusion model with public style or domain models. Experiments on SD 1.5 show that erased concepts can reappear both when merging weakens the erasure update and when partner updates are added without reducing it. Strength-tuned addition provides a better suppression–appearance trade-off than interpolation, while the tested parameter guards offer limited additional gains.

Critical strengths

The paper usefully separates attenuation of the erasure update from partner injection and examines the trade-off between concept suppression and appearance transfer. The setting is interesting.

Critical weaknesses

W1: Limited novelty and mechanistic insight. The more interesting finding is that partner injection can restore target concepts while keeping the erasure coefficient fixed, but the analysis provides limited insight into why this happens.

W2: The practical motivation needs stronger justification. The partner already possesses the ability to generate the target concept, so its reappearance after merging is not inherently surprising. In the stated benign composition setting, merging first and then applying erasure appears to be a natural alternative. Indeed, Interpolation + UCE-EOS satisfies both reporting thresholds in 32/33 settings, although at reduced appearance transfer. The paper does not clearly establish when composing an already-erased model offers a practical advantage over this alternative workflow.

W3: Limited experimental coverage and evaluation reliability. Experiments cover only SD 1.5 and three benign ImageNet objects, leaving applicability to NSFW content, artistic styles, and personal identities unclear. Each setting also uses very few prompt templates and only three or four generation seeds. Evaluation relies on automated classifier and CLIP scores without human validation; agreement between these metrics does not establish accurate concept detection or perceptually meaningful appearance transfer.

W4: The presentation obscures the main conclusions. Repeated qualifications such as “does not establish” and “not evidence,” together with references to “historical,” “archived,” and “legacy” results, make the experimental narrative difficult to follow. Acknowledging limitations is appropriate, but the current organization makes it hard to distinguish the principal findings from implementation history and caveats.

W5: Minor presentation issues. Figure 4’s caption lacks a final period, and Section 5.2 uses inconsistent numerical precision. The symbols in Eq. (8) should be explicitly defined. The attention guard should be described in terms of cross-attention layers, with the implementation-specific attn2 naming retained only as clarification.

Questions for authors

Q1: What explains recovery when the erasure coefficient remains fixed? Since the partner can already generate the target concept, recovery could reflect information transferred from the partner, disruption of the existing suppression mechanism, or both. What evidence distinguishes these explanations, and what does the analysis reveal beyond the observation that adding a target-capable partner can restore target generation?

Q2: Under what practical constraints is merging an already-erased model the preferred workflow? Given the strong suppression achieved by re-erasure in Table 2, please clarify when merging first and then erasing is unavailable or undesirable, and how the proposed workflow balances suppression, appearance transfer, and required access.

Q3: What is the actual computational advantage of avoiding re-erasure? Table 2 reports composition timers rather than comparable end-to-end costs. How much additional time, data, and implementation access does re-erasure require in the evaluated setting, and are these requirements substantial enough to motivate the training-free alternatives?

Reviewer_rgRUOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper asks what happens to an erased model when you merge it with another fine-tuned checkpoint from the same base model (e.g. "anime" checkpoint). The authors treat each checkpoint as “base model + update”: the erased model is base + erasure update, and the other checkpoint is base + partner update. They compare four ways of merging:

  1. Averaging: base + a × erasure update + (1 − a) × partner update

  2. Adding: base + full erasure update + λ × partner update

  3. TIES: base + a trimmed sign-matched merge of the two updates.

  4. Guarded adding: same as Adding, except the overlap between partner update and erasure update are removed.

The main finding is that averaging brings back the erased concept. Adding preserves most of the erasure. The guards help by only about 1 point. Re-running the erasure on the merged model works best of all.

Critical strengths
  1. The main underlying question is very practical and understudied. I believe merging the checkpoints has started to gain traction in the community, and an erasure or an edit that does not survive this does seem like a real deployment concern.

  2. Even though it was hard to understand and comprehend, the paper provides thorough analysis and implementation details for reproduction.

Critical weaknesses
  1. The paper is clearly completely AI-written. It seems like there was no human verification done post AI writing. The paper is very hard to read, to the point that I could not follow the main story and had to read it multiple times, over and over again, to even try to understand what was happening. I strongly believe the authors need to carefully rewrite the paper so that it is easier for a reader to understand the point.
  • The abstract uses terms like "partner-appearance transfer", "common minimum appearance-transfer targets", "normalized suppression retention", "parameter guards", "the measured appearance estimator". It is difficult to understand what those mean. Reading the abstract for the first time, a reader will not be able to understand what was done and what was found in the paper.

  • Similarly, the introduction is very dense and more complicated to read. It sounds like a list of caveats rather than a motivation or building up the actual work. The reader is introduced to terms and words that were not talked about before.

  • Figure 1 does not add much value to the paper if it's not properly annotated. The "Score" is never defined. The notations are used before even defining them. The word "partner" is not explained.

  • The results section just reads like a stream of numbers with changing meanings and denominators, and I found it very hard to follow. In a single paragraph, the reader is introduced to multiple scores (0.00145, 0.14445, 0.21242, 0.000445).

  1. The two merging types of Interpolation and addition can be narrowed down to the same equation θb+aδE+bδT\theta_b + a \delta_E + b \delta_T. So the real claim is "dont scale the erasure update below 1" instead of "addition beats interpolation". When you view it this way, task vectors are composed in the same way (Ilharco et al. (ICLR 2023)). Kim et al. (arXiv 2412.00357) even built W0+ΔWsafe+ΔWftW_0 + \Delta W_{safe} + \Delta W_{ft} and showed that the erased concept is best protected when such scaling is done, but not completely. The real contribution is in using pubic checkpoints and the matched-appearance comparison, which is too thin for ICLR.

  2. The guards assume the actual erasure happens in delta_E's own layers or subspaces. Since ESD-u trains everything other than cross attention layers (e.g. 4D conv layers), how's does the guard that applies only to 2D matrices work here? the row-space guard only works on 2D matrices, and the attention guard works only cross-attention, which ESD-u never changes, yet it reaches 99.6% retention on average. If this is true per eraser, then the guard is working by blocking the partner's cross attention, not by protecting the delta_E. The paper does not discuss these nuances.

  3. The main experiments are run only on three ImageNet classes on SD 1.5 only. Adding at least the 10 imagenette classes and a few models would strengthen the experimental claims. There are no styles, identities or NSFW concepts, despite the safety framing. So adding those will also help.

  4. The main title of the paper suggests that the concept is still present in the erase model and it's not really gone, and that merging uncovers it. But it seems like every merged model, or as the authors call it, the partner model, can still draw the erased concept from the original model. The partner alone scores highly on the erased church concept, and even the attenuation-only control moves towards the base model, which draws church fine. Technically, this result can be seen in two ways:

  • The partner model supplies the erased concept again.
  • The partner update breaks the erasure and uncovers what the erasure model still knows.

I feel like the first one needs to be properly tested before making such claims. If not, the whole argument falls apart.

Questions for authors
  1. Why is the "Robust Concept Erasure Using Task Vectors." (Pham et al.) method not used in this paper. Their erasure method matches really closely with the paper's algebra.

  2. How do the results split by different erasure methods? I'm particularly interested if the original UCE method resists addition the way it resists fine-tuning, as shown in prior work.

  3. If the partner model cannot produce the target concept, does the merging still restore it? This is the experiment that separates the claim of the past (the partner just reteaching the concept) from whether the partner actually breaks the erasure. I think this is a very necessary experiment for most of the claims.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.