acceptodds
Withdrawn from ICLR 2027

Deep Localized Concept Erasure via Visually-Cued Bregman Preference Optimization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Concept erasure in text-to-image diffusion models demands both spatial locality to preserve benign contexts and deep removal to withstand adversarial reactivation. Existing methods fall short on the two dimensions: text-centric approaches leave undesirable visual features intact, while image-centric preference learning (e.g., DPO) suffers from global entanglement and gradient vanishing, resulting in superficial marginal ranking rather than persistent erasure of undesirable concepts. To overcome these bottlenecks, we propose a Visually-Cued Bregman Preference Optimization framework for deep localized concept erasure. First, we enforce strict spatial locality by grounding preference optimization on masked, minimally-edited safe-unsafe image triplets. Second, to achieve deep erasure, we deliberately expose visual cues of undesirable concepts to the model during fine-tuning, forcing the model to thoroughly erase the concept under strong adversarial visual stimuli. Finally, we further propose to use Bregman Preference Optimization (BPO) for concept erasure and generalize it to diffusion models. We analytically demonstrate that Diffusion BPO provides non-vanishing optimization signals, sustaining continuous concept suppression well beyond the limitations of DPO. Extensive experiments demonstrate that our framework achieves superior erasure efficacy, robust resistance against optimized textual and multimodal reactivation attacks, and high-fidelity locality preservation compared to representative baselines.

Primary area:
generative models
Keywords:
Concept Erasure, Text-to-Image Diffusion Models, Machine Unlearning

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_p9tqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes localized concept erasure through masked preference learning on minimally edited safe–unsafe image pairs. Residual image features injected through a frozen IP-Adapter provide target-related visual cues, and a BPO objective replaces DPO. Visual erasure and ESD-style semantic cleanup update separate self-attention and cross-attention LoRAs. Experiments primarily evaluate nudity erasure on SD v1.4, with additional concepts and SDXL/FLUX extensions.

Critical strengths
  1. Meaningful combination of localized supervision and visual cues. In Table 4, removing localized triplets or residual injection increases UnlearnDiffAtk ASR from 6.34% to 33.80% or 24.65%, supporting their practical value.

  2. Strong reported erasure–preservation trade-off on SD v1.4. Table 1 reduces I2P detector-positive outputs from 337 to 1 while reporting FID 13.12 and LPIPS 23.67. Table 8 further supports suppression across four image-conditioning interfaces.

Critical weaknesses
  1. The gradient explanation does not fully describe the implemented objective. Although the unclipped loss has a non-vanishing margin derivative, Algorithm 1 uses hard clipping, which gives zero derivatives outside the clipping interval. Clipping frequency is unreported.
  2. The target-specificity of the residual cue is insufficiently justified**

Eq. (4) define cΔimgc_\Delta^{\mathrm{img}}. However, feature subtraction does not ensure that cΔimgc_\Delta^{\mathrm{img}} isolates target-specific information: the residual may also encode replacement content, editing artifacts, or changes in non-target attributes.

Questions for authors
  1. Please specify hyperparameters β,τ,λbg,γ\beta,\tau,\lambda_{\mathrm{bg}},\gamma. What fraction of training samples is clipped, and does this support the persistent-gradient explanation?
  2. What evidence supports the claim that the residual cue in Eq. (4) isolates target-specific visual information?
1 comment

We thank Reviewer p9tq for recognizing our localized supervision, residual visual cues, and erasure–preservation trade-off. Below, we will address each of your concerns in detail.

Q1: Hyperparameters, clipping frequency, and the persistent-gradient explanation

Hyperparameters. For SD v1.4, β\beta scales the preference margin, τ\tau bounds it before exponentiation, λbg\lambda_{\mathrm{bg}} weights background preservation, and γ\gamma controls target-word cleanup strength:

Categoryβ\betaτ\tauλbg\lambda_{\mathrm{bg}}γ\gamma
Nudity / Celebrity50611
IP characters560.51
Objects560.12.5

Clipping frequency. As you noted, hard clipping m~=clip⁡(m,−τ,τ)\tilde m=\operatorname{clip}(m,-\tau,\tau) gives zero preference gradient outside the clipping interval. We measured the fraction outside either bound in a separate 1,200-step visual-only BPO run with residual cues, with clipping enabled throughout:

Training updatesClipped margins (%)
1–4000.00
401–8007.25
801–120088.50
Overall31.92

Substantial late-stage clipping does not support persistent gradients throughout training. Within the unclipped interval, DPO and SBA-BPO multiply the negative margin gradient by

gD(m)=11+em,gB(m)=e−2m+em4.g_D(m)=\frac{1}{1+e^m},\qquad g_B(m)=\frac{e^{-2m}+e^m}{4}.

DPO's multiplier decays as the positive margin grows. SBA-BPO's remains positive and increases for m>ln⁡(2)/3m>\ln(2)/3; it is not monotonically increasing over all positive margins. Clipping stabilizes this mechanism; it does not certify complete erasure of clipped samples.

We will document the settings and clipping statistics and specify the unclipped regime.

Q2: Evidence that residual cues are more target-focused

Our claim is stronger target focus relative to full-image features, rather than complete semantic isolation. Paired subtraction is designed to reduce shared-scene information.

(1) Quantitative target-focus diagnostic. Inspired by WinCLIP's local similarity mapping [1] and RISE's deletion-based evaluation [2], we test target–background separation and the response drop after target removal. Across 16 reference pairs and 20 disjoint Pikachu test pairs, the encoder, crops, and masks are fixed. Responses are cosine similarities; the gap compares target and background means, AUROC ranks target pixels, and removal drop weights matched-crop response decreases by target coverage.

Visual representationTarget–background gap ↑Target-pixel AUROC ↑Target-removal drop ↑
Full-image feature0.12000.82730.3743
Residual feature ΔH\Delta H0.17720.87340.5667

Residual cues improve all three indicators, including a 47.7% larger target–background gap. This supports stronger target focus in the tested encoder space, rather than complete semantic isolation.

See Figure 2 in the revised main text and Appendix A.3.3 for visualizations and quantitative diagnostics.

(2) Why the cue need not directly penalize replacement content. As in Eq. (6), the residual cue conditions only the dispreferred unsafe branch, while the preferred safe branch is evaluated with a null cue on the safe latent. This safe branch anchors the replacement image. This asymmetry addresses the concern but does not guarantee preservation of every replacement attribute.

(3) Direct conditioning ablation and training benefit. With BPO fixed, we compared full-image and residual cues in 1,200-step visual-only training without target-word cleanup. IP-Adapter detections use 135 sources; benign LPIPS uses 64 matched COCO inputs:

Visual cueTarget detections ↓Benign-image LPIPS×100 ↓
Full-image217.60
Residual013.94

In the full framework, removing residual injection also raises UnlearnDiffAtk ASR from 6.34% to 24.65%. These results support the target focus, robustness, and preservation benefits of residual cues; we will replace the purity claim accordingly.

References

[1] J. Jeong et al. WinCLIP: Zero-/Few-Shot Anomaly Classification and Segmentation. CVPR, 2023.

[2] V. Petsiuk, A. Das, and K. Saenko. RISE: Randomized Input Sampling for Explanation of Black-box Models. BMVC, 2018.

Reviewer_NB5uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes localized concept erasure using masked, minimally edited unsafe–safe image pairs. It injects a residual visual cue during training, applies Bregman Preference Optimization (BPO) to a diffusion preference objective, and separately weakens text–concept associations. Most experiments focus on nudity erasure in SD v1.4, with additional SDXL and FLUX results.

Critical strengths
  1. Training with target-related visual cues leads to low success rates for optimized text attacks and several image-conditioning attacks on SD v1.4.

  2. The paper evaluates many baselines and reports erasure, robustness, image quality, and content preservation rather than erasure alone. Matched image pairs, masked losses, and off-mask feature preservation provide a coherent way to limit changes outside the target region; BPO adds a measurable improvement over the paper’s DPO variant.

Critical weaknesses
  1. GLoCE(Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptation) already performs localized concept erasure, AlignGuard(AlignGuard: Scalable Safety Alignment for Text-to-Image Generation) applies DPO to harmful–safe image pairs, and TICoE(Beyond Text Prompts: Precise Concept Erasure through Text-Image Collaboration) incorporates visual guidance into erasure. Given these precedents, the paper does not clearly establish substantial novelty beyond replacing the DPO loss with an existing BPO objective. The methodological distinction of its residual visual cues and masked preference comparisons requires further clarification.

  2. The strongest results come from U-Net-based models, while DiT evaluation remains limited in backbone coverage and reactivation testing. More comprehensive experiments on FLUX and additional backbones such as PixArt are recommended to assess whether erasure efficacy, preservation, and robustness generalize across architectures.

Questions for authors
  1. The paper trains one adapter per concept and shows only a qualitative composition of three adapters, without measuring joint erasure or interference. How do joint erasure, preservation, and cost scale when composing adapters for 10, 20, or 50 concepts?

  2. How does the method perform when simultaneously erasing a larger number of concepts (e.g., 10, 20, or 50), in terms of erasure efficacy and preservation of untargeted concepts?

  3. Could you evaluate additional DiT-based models, such as PixArt, and conduct more comprehensive experiments on FLUX?

1 comment

We thank Reviewer NB5u for recognizing our robustness and localized preference design. Below, we will address each of your concerns in detail.

Q1: Methodological distinctions from GLoCE, AlignGuard, and TICoE

Our contribution extends beyond loss choice along three axes:

  1. Vs. GLoCE: GLoCE uses gated low-rank adaptation. We localize generative supervision using pixel-aligned safe–unsafe triplets, masked denoising comparisons, and off-mask feature anchoring.
  2. Vs. AlignGuard: AlignGuard applies DPO to harmful–safe pairs. Our asymmetric visual-cue conditioning challenges only the unsafe branch; the safe image anchors desired content, while BPO changes margin weighting.
  3. Vs. TICoE: TICoE uses text–image guidance. Our difference-of-features residual cues subtract safe-image features from unsafe-image features to reduce shared-scene information in the visual challenge.

Q2: Scaling to 10, 20, and 50 concepts: composition, preservation, and cost

We have not yet evaluated 20/50-concept composition or joint training due to time and compute limits. We evaluate simultaneous erasure of ten concepts below.

We combine independently trained adapters for Nudity, Taylor Swift, Elon Musk, Pikachu, Snoopy, Mickey Mouse, SpongeBob, Hello Kitty, Airplane, and Church into one rank-8 adapter without retraining using Compress then Merge (CtM) [1]. This single model is Merged model (10).

(1) Target suppression. Counts are target absences on 200 matched inputs per target, including Base negatives; unknowns are not successes. Nudity uses six-class NudeNet; other targets use Qwen3-VL-8B.

Target conceptBase: absent /200 ↑Merged model (10): absent /200 ↑
Nudity72194
Taylor Swift38191
Elon Musk75188
Pikachu0191
Snoopy9180
Mickey Mouse14126
SpongeBob30192
Hello Kitty6191
Airplane687
Church1103

Seven targets achieve 90–97% non-detection, giving us confidence in further scaling. Airplane/Church show interference: single-adapter counts 185/200 and 183/200 fall to 87/200 and 103/200 after merging; Mickey Mouse is weaker (126/200).

(2) Benign-generation utility. We use 1,000 matched COCO inputs; image–text CLIP uses ViT-B/32.

ConceptsModelCOCO1K FID ↓COCO1K CLIP ↑
0Base63.3531.31
10Merged model (10)77.9530.61

FID rises by 14.60 and CLIP falls by 0.70, indicating a benign-utility cost. Named untargeted-concept retention remains untested; COCO1K measures aggregate utility.

Computational cost. Ten-concept deployment uses one 1.59M-parameter adapter; training costs accumulate per concept. Costs at 20/50 concepts remain unmeasured.

Qualitative examples are in Appendix D.4.

Q3: Additional DiT backbones and FLUX reactivation evaluation

We extend evaluation to FLUX.1-dev and PixArt-Σ-512.

(1) FLUX suppression and recovery. NudeNet counts nudity detections; MMA uses fixed adversarial prompts without model-specific optimization. LPIPS uses 4,703 matched original/edited I2P outputs.

ModelI2P /4703 ↓MMA /1000 ↓LPIPS ×100 ↓
Original483492—
GEM16814753.49
Ours23418237.86

GEM suppresses more; ours has lower LPIPS at the reported operating points. The modified ReFlux-inspired test uses eight references and 200 training updates. On 58 common Original-positive inputs, detections before→after are 58→46 (Original), 11→19 (GEM), and 14→19 (Ours). Both permit recovery but remain below the attacked Original.

(2) PixArt-Σ-512. On 32 custom prompts ×2 seeds, detections fall 46→0/64. On 64 unrelated COCO captions, CLIP changes 27.27→27.13. This pilot supports transfer; recovery testing remains limited to FLUX.

Qualitative examples are in Appendix D.6.

References

[1] Z. He et al. Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter. 2026.

Reviewer_XAoROpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces Visually-Cued Bregman Preference Optimization, an preference framework for concept erasure. The authors first curate a triplet dataset for concept erasure. Each triplet includes an image containing the target concept, a mask of the target concept region, and the image without the target concept. The authors use the mask for localized preference loss. The authors claim the operational definition of the text-to-image model being erased of the target concept is that the model cannot generate a target concept image even when we provide strong visual cues of the target concept. Therefore, the authors then use the frozen IP-Adaptor to provide the visual cue, which is the residual feature between the image with and withour the target concept, of the target concept during training. Moreover, the authors introduce Bregman Preference Optimization (BPO) to text-to-image diffusion models by using a masked denoising surrogate to approximate the density ratio in BPO. The authors also use legacy semantic learnup and decoupled optimization in their framework.

Critical strengths
  1. The authors curate a triplet dataset for concept erasure. This dataset will benefit the research community if the authors publish it. However, I also notice some works[a,b] that also curate similar datasets.

  2. To the best of my knowledge, this is the first work introducing BPO into text-to-image models.

Ref:

[a] Yu, Zhenyu, et al. "Forgetme: Benchmarking the selective forgetting capabilities of generative models." Engineering Applications of Artificial Intelligence 161 (2025): 112087.

[b] Xiang, Qianlong, et al. "VisualErase: Dual-Branch Visual Trajectory Redirection for Robust Concept Erasure in Text-to-Image Diffusion Models." arXiv preprint arXiv:2610.05000 (2026).

Critical weaknesses
  1. Lack of support.

    a . The authors claim that relative preference optimization falls short of deep erasure. (L57) However, the authors still use BPO, which is also a relative preference optimization.

    b. The authors claim that the intrinsic marginal ranking of DPO is caused by gradient vanishing. (L71-72) This observation is not supported by any experiment or related work.

    c. The key idea and claim of the human memorization work are not supported, especially in cognitive science. (L193-199)

  2. Insufficient experiments.

    a. Table 1 lacks MS-SSIM and CLIP score, although I found them in Table 6.

    b. Ablation studies only provide the results without a single component, making it hard to determine the relationship between each component.

    c. This paper mainly focuses on BPO, but the contribution of BPO is mainly on Preservation (MS-SSIM).

  3. The overall picture of the frame is unclear. This paper discusses each component but lacks an overview introduction of the whole framework.

Questions for authors
  1. Why do the authors still use BPO, which is also a relative preference optimization?

  2. Why is the intrinsic marginal ranking of DPO caused by gradient vanishing?

  3. Why does Table 1 lack MS-SSIM and CLIP score?

  4. Do the key idea and claim of the human memorization work have any support?

1 comment

We thank Reviewer XAoR for recognizing our curated triplet dataset. Below, we will address each of your concerns in detail.

Q1: Why use BPO if it is also a relative preference optimization?

BPO is also a relative preference objective. We retain safe–unsafe comparisons but strengthen their conditioning and margin weighting:

  • Why standard relative preference falls short: DPO optimizes −log⁡σ(m)-\log\sigma(m), where m>0m>0 favors the safe image. Its gradient multiplier σ(−m)\sigma(-m) decays as the margin grows, weakening further optimization after favorable ranking.
  • Why retain a preference-based formulation: Safe ywy_w anchors desired content; unsafe yly_l defines what to suppress. Favorable ranking alone does not certify complete erasure.
  • How our framework strengthens erasure: (1) Asymmetric Visual-Cue Conditioning: residual cues enter only the unsafe branch, encouraging safe preference despite visual hints. (2) SBA-BPO Objective: its multiplier (e−2m+em)/4(e^{-2m}+e^m)/4 increases for m>ln⁡(2)/3m>\ln(2)/3, maintaining pressure within the unclipped interval. Hard clipping gives zero preference gradient outside that interval, limiting the non-vanishing-gradient claim.

Q2: Support for the claim relating DPO ranking and gradient vanishing

We will correct this causal wording: pairwise comparison defines ranking; gradient attenuation weakens further margin optimization. Evidence comprises analysis, theory, and controlled experiments:

  1. Analytical proof: Eq. (9) gives ∇θℓ=−g(m)∇θm\nabla_\theta\ell=-g(m)\nabla_\theta m. DPO has gD(m)→0g_D(m)\to0 as m→∞m\to\infty; the parameter gradient also depends on ∇θm\nabla_\theta m.
  2. Related work: Rafailov et al. (2023) formulate DPO's logistic objective; Kim et al. (2025) introduce the BPO formulation we adopt.
  3. Empirical support: The controlled comparisons in Q5 support BPO's benefit, without proving saturation is the sole cause of incomplete erasure.

Q3: Human memorization analogy in method

We do not provide cognitive-science evidence that failed recall implies complete deletion. The human-memory discussion is an intuitive analogy motivating recovery tests under strong visual cues. Across four image-conditioning interfaces on 135 sources, any-interface ASR is 3.70% for ours versus 61.48% for DUO. This supports resistance to the tested reactivation attempts, without certifying complete deletion. We will use machine-learning terms and discuss ForgetMe and VisualErase.

Q4: MS-SSIM and I2I CLIP in the main results

We will merge MS-SSIM and I2I CLIP into Table 1 in the revised manuscript. On 4,703 matched pairs, our LPIPS/MS-SSIM/I2I CLIP are 23.67/79.97/89.25 (×100). I2I CLIP measures image similarity; COCO image–text CLIP measures prompt alignment.

Q5: Ablation study design and the contribution of BPO

BPO beyond MS-SSIM: With all other components fixed, BPO also improves erasure and adversarial robustness:

ConfigurationI2P Unsafe Count ↓MS-SSIM ×100 ↑UDA ASR (%) ↓
DPO (w/o BPO)373.799.15
Ours (BPO)179.976.34

DPO already uses localized triplets and residual cues; BPO lowers UnlearnDiffAtk ASR by 2.81 points (30.7% relative).

Controlled component relationships. Beyond the leave-one-out ablation, four 1,200-step visual-only runs cross loss and cue design without target-word cleanup. IP-Adapter recovery uses 135 target-positive sources; LPIPS uses 64 benign COCO inputs.

ConfigurationTarget detections ↓\downarrowLPIPS ×100 ↓\downarrow
DPO + Full-image cue478.18
BPO + Full-image cue217.60
DPO + Residual cue197.31
BPO + Residual cue013.94

BPO strengthens suppression with either cue; residual cues improve suppression and preservation with either loss. They are complementary; BPO alone does not improve preservation here.

Q6: Overview introduction and architecture diagram

The framework has four steps:

  1. Triplets: aligned unsafe–safe pairs and masks define erasure and preservation targets.
  2. Residual cues: subtract safe from unsafe image features; a frozen IP-Adapter conditions only the unsafe branch.
  3. Visual erasure: masked BPO updates self-attention LoRAs; off-mask anchoring limits background changes.
  4. Cleanup and inference: alternating updates to separate cross-attention LoRAs weaken target-word associations. Inference retains both LoRAs without the IP-Adapter, image cue, or mask.
Reviewer_EpeWOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a visually-cued Bregman Preference Optimization (BPO) framework for deep and localized concept erasure in text-to-image diffusion models. The method uses masked safe-unsafe image pairs to focus unlearning on target regions and introduces residual visual cues to improve resistance to concept reactivation. It further adopts Diffusion BPO to address the gradient saturation of conventional DPO, along with a semantic cleanup objective to weaken text–concept associations. Experiments demonstrate strong erasure effectiveness, robustness against textual and image-conditioned attacks, and preservation of unrelated visual content, with additional evaluations on SDXL and FLUX

Critical strengths

1. Interesting framework for robust and localized concept erasure

The paper tackles the challenging problem of removing undesirable concepts while preserving unrelated visual content. I find the use of visual cues during unlearning particularly interesting, as it provides a way to strengthen concept suppression beyond text-based supervision. The integration of localized preference learning and BPO also makes the overall framework well motivated

2. Convincing empirical results and extensive evaluation

The experimental results are strong across multiple aspects, including erasure effectiveness, adversarial robustness, and preservation. In particular, the evaluation against both textual and image-conditioned attacks provides useful evidence of the method's robustness. The ablation studies are also well organized, and the additional results on SDXL and FLUX help demonstrate the applicability of the approach to different model architectures

Critical weaknesses

1. Limited evaluation of spatial locality in the main nudity setting

I appreciate that the paper evaluates preservation using several metrics and reports background LPIPS for other concepts. However, the main nudity evaluation relies on whole-image similarity across all 4,703 I2P prompts, while only 337 outputs from the original SD v1.4 model are detected as unsafe. I wonder why the number of detected unsafe images is relatively small and whether this is related to the prompt distribution or the detection criteria. More importantly, I am not sure whether the aggregate preservation scores sufficiently capture spatial locality when the target concept is actually present. Evaluating background preservation specifically on these target-positive samples would provide more direct evidence for localized erasure

2. Limited evaluation of multi-concept scalability

Although Appendix D.8 demonstrates the composition of three independently trained concept adapters, the evaluation is limited to qualitative examples. I think multi-concept scalability is particularly important from a practical perspective, as real-world applications often require removing multiple undesirable concepts rather than a single one. However, it remains unclear whether combining more adapters would introduce interference that compromises erasure effectiveness or preservation. A quantitative evaluation with an increasing number of erased concepts would help demonstrate the practical applicability and scalability of the proposed framework.

Questions for authors

Please see the weaknesses above. Additionally, how could the proposed framework handle concepts with ambiguous spatial boundaries, such as artistic styles, where reliable concept masks are difficult to define? Since the method relies heavily on spatially localized masks, I am concerned that its generalizability may be limited to concepts with clearly identifiable regions. Although this limitation is acknowledged in Appendix E, I would appreciate further discussion of how the method could be extended to more general concepts!

+) I also suggest adding a content warning for potentially sensitive visual content, as several figures contain nudity-related imagery (e.g., Figures 2, 3, 5, 12, and 13), while Figure 14 includes graphic depictions of violence and blood

1 comment

We thank Reviewer EpeW for recognizing our visual cues and localized preference framework. Below, we will address each of your concerns in detail.

Q1: Why 337/4,703 outputs are detected as unsafe, and background preservation on the target-positive subset

Why 337 out of 4,703 I2P images are detected as unsafe. This stems from both the prompt distribution of I2P and the NudeNet evaluation criterion. First, I2P covers seven broad inappropriate categories; only a subset of prompts is sexually explicit. Second, an image is flagged when any of the nine evaluated exposed-body classes exceeds confidence 0.6. Under this criterion, unedited SD v1.4 yields 337 positives.

Background preservation on the target-positive subset. We evaluate background preservation on all 337 target-positive images. All methods share prompts, seeds, and original-image masks; identical gray fill excludes the person region. Preservation scores are ×100.

MethodUnsafe Count /337 ↓Background LPIPS ↓Background MS-SSIM ↑
DUO6216.4488.91
GLoCE248.6553.52
Ours114.2490.05

Our method suppresses 336/337 positives with the best background scores here, supporting locality during active erasure. These metrics exclude attributes inside the person region.

Q2: Quantitative evaluation of multi-concept scalability

We add ten-concept quantitative results; a systematic concept-count scaling study remains incomplete.

We combine independently trained adapters for Nudity, Taylor Swift, Elon Musk, Pikachu, Snoopy, Mickey Mouse, SpongeBob, Hello Kitty, Airplane, and Church into one rank-8 adapter without retraining using Compress then Merge (CtM) [1]. This single model is Merged model (10).

(1) Target suppression. Each target uses 200 matched inputs. Counts denote target absence, including Base-negative cases; unknown judgments are not successes. Nudity uses six-class NudeNet; other targets use Qwen3-VL-8B.

Target conceptBase: absent /200 ↑Merged model (10): absent /200 ↑
Nudity72194
Taylor Swift38191
Elon Musk75188
Pikachu0191
Snoopy9180
Mickey Mouse14126
SpongeBob30192
Hello Kitty6191
Airplane687
Church1103

Seven concepts reach 90–97% non-detection in one model. Interference is substantial for Airplane/Church: 185/200 and 183/200 with individual adapters fall to 87/200 and 103/200 after merging. Mickey Mouse is weaker (126/200). Most targets remain strongly suppressed, with uneven composition interference.

(2) Benign-generation utility. We evaluate image quality and prompt alignment on 1,000 matched COCO inputs; CLIP uses ViT-B/32.

ConceptsModelCOCO1K FID ↓COCO1K CLIP ↑
0Base63.3531.31
10Merged model (10)77.9530.61

FID rises by 14.60 and CLIP falls by 0.70: prompt alignment is largely retained, with a generation-quality cost.

Qualitative examples of three- and ten-concept composition are in Appendix D.4.

Q3: Handling concepts with ambiguous spatial boundaries

For diffuse or global concepts such as artistic styles, two possible extensions are (1) a full-image mask, with content-matched style pairs and a content-preservation objective replacing off-mask anchoring, and (2) soft masks, weighting uncertain boundaries continuously. These extend the formulation beyond hard localized masks but remain unvalidated. We will expand Appendix E to distinguish these possibilities from demonstrated capabilities.

Q4: Content warning

The revised paper includes a content warning immediately after the abstract for nudity and graphic violence.

References

[1] Z. He et al. Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter. 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.