acceptodds
Under review as a conference paper at ICLR 2027

SHARE: Rectified Half-Space Preservation for Balancing Concept Erasure and Retention

Abstract

Editing-based concept erasure removes specified concepts from text-to-image diffusion models while preserving non-target behavior. However, strict preservation constraints in closed-form parameter editing can excessively restrict the available edit space. We formulate editing-based erasure as a constrained optimization problem and show that equality-based preservation limits the feasible degrees of freedom. We then introduce a rectified half-space relaxation that makes retained representations inactive without requiring their preactivations to vanish. Based on this formulation, we propose Set-aware Half-space with Analytic Readouts for Erasure (SHARE). SHARE learns a shared rectified selector and computes layer-specific residual readouts in closed form, enabling concept-dependent updates without jointly optimizing the selector and readouts. Experiments across multiple concept scales, prompt types, and model architectures demonstrate favorable trade-offs between target erasure and non-target retention.

Primary area:
generative models
Keywords:
Concept erasure, Diffusion models, Unlearning

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_2VK4OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SHARE, a concept erasure method that addresses the limitations of equality-based preservation constraints in existing closed-form editing approaches, which restrict the available editing space. In Stage 1, SHARE learns a rectified selector to preserve erasure-relevant information while suppressing activations on retained concepts. In Stage 2, it computes layer-specific residual readouts in closed form. Extensive experiments demonstrate competitive erasure-preservation trade-offs across various concept erasure settings.

Critical strengths
  • The paper analyzes the structural limitations of equality-based preservation constraints and proposes a rectified half-space formulation to alleviate these restrictions.
  • The proposed method demonstrates competitive performance across multi-concept, implicit and instance/style erasure settings.
  • The paper provides extensive ablation studies on loss components, retained-set coverage, and representation dimension, supporting the effectiveness of the proposed design.
Critical weaknesses

[W1] Unclear separation between training and evaluation data. It is unclear whether the retain set used during training is independent of the final evaluation data. In addition, the construction of the validation set used for hyperparameter selection and its separation from the final evaluation data are not sufficiently specified.

[W2] Limited hyperparameter sensitivity analysis across concept domains. Although the representation dimension and inference scaling factor vary across concept domains, the corresponding sensitivity analyses are primarily conducted in the celebrity erasure setting.

Questions for authors
  1. Could the authors clarify whether the retain set used during training is independent of the final evaluation data? Additionally, how is the validation set for hyperparameter selection constructed, and is it separate from the final evaluation data? If there is any overlap, could the authors provide additional results using an independent evaluation set?

  2. Does preservation using a predefined retain set generalize to unseen concepts and different generation capabilities? For example, when MS-COCO prompts are used as the general retain set, can compositional generation capabilities measured by GenEval[1] also be preserved, or does the retain set need to be tailored to specific capabilities?

  3. Does preservation performance in generated images improve as the size of the retain set increases, and how does this affect erasure performance? Comparing SHARE with SPEED across different retain set sizes would help clarify the practical benefits of the proposed half-space relaxation.

  4. While SHARE demonstrates effective multi-concept erasure in the celebrity setting, does it achieve similarly favorable erasure-preservation trade-offs when erasing multiple concepts in other domains, such as characters or artistic styles?

  5. Why is a substantially larger inference scaling factor used for nudity erasure compared to other concept domains?

  6. How robust is the final SHARE model against adaptive white-box attacks, such as UnlearnDiffAtk[2]? It would be valuable to evaluate whether the erasure effectiveness is maintained under such attacks.

  7. Given the importance of generalization to modern DiT/flow-based architectures, could the authors extend the FLUX.1-dev evaluation beyond I2P to include adversarial benchmarks? Such results would provide a more comprehensive assessment of SHARE's effectiveness and robustness on recent architectures.


[1] Ghosh, Dhruba, Hannaneh Hajishirzi, and Ludwig Schmidt. "Geneval: An object-focused framework for evaluating text-to-image alignment." NeurIPS, 2023.

[2] Zhang, Yimeng, et al. "To generate or not? safety-driven unlearned diffusion models are still easy to generate unsafe images... for now." ECCV, 2024.

7 comments
AuthorsOpenReview ↗

Response to Reviewer 2VK4 (1/3)

We thank the reviewer for the detailed and practically important questions. We address the shared concerns regarding data separation and adaptive white-box robustness in [G-1] and [G-4] of our General Response. Below, we report additional evaluations of cross-distribution retention, retained-set coverage, and hyperparameter sensitivity. The cross-domain multi-concept and FLUX.1 adversarial evaluations remain in progress.


[2VK4-1] (W1, Q1) Final evaluation prompts are held out from training and validation.

Unclear separation between training and evaluation data.

Could the authors clarify whether the retain set used during training is independent of the final evaluation data?

Training, validation, and final evaluation prompts are disjoint. As detailed in [G-1], celebrity identities are shared across stages following the benchmark protocol, while prompt templates are disjoint. General MS-COCO training and evaluation captions are also disjoint.

These experiments evaluate generalization to unseen prompt formulations for the same identities by following the OCE protocol. We have documented the complete splits in the revised appendix and included machine-readable prompt lists in the updated supplementary material.


[2VK4-2] (W2) We have extended the sensitivity analysis regarding rr and γ\gamma to the implicit concept erasure case.

Limited hyperparameter sensitivity analysis across concept domains.

the corresponding sensitivity analyses are primarily conducted in the celebrity erasure setting

[TL;DR] In implicit concept erasure, increasing representation dimension rr or inference scaling γ\gamma improves erasure at the cost of greater image deviation. Performance largely saturates around r=16r=16 and γ=8\gamma=8–1212. We have added both sensitivity analyses to Appendix E.6.

Representation dimension (rr).

We vary rr without adversarial refinement.

rrI2P ↓\downarrowMMA ↓\downarrowRAB ↓\downarrowCLIP ↑\uparrowFID ↓\downarrow
40.320.190.0126.492.12
80.200.170.0026.482.57
16 (default)0.120.120.0026.494.05
320.120.120.0026.475.68
640.120.060.0026.456.89

Erasure improves from r=4r=4 to r=16r=16 and then largely saturates, apart from a further MMA reduction at r=64r=64. FID increases steadily, while CLIP Score remains nearly unchanged.

Inference scaling factor (γ\gamma).

Using one adversarially refined checkpoint, we vary only γ\gamma.

γ\gammaI2P ↓\downarrowMMA ↓\downarrowRAB ↓\downarrowCLIP ↑\uparrowFID ↓\downarrow
0 (original SD v1.4)0.830.601.0026.50—
10.370.390.5826.511.47
40.080.090.0026.492.88
80.020.050.0026.514.17
100.020.040.0026.504.60
12 (default)0.030.020.0026.504.82
140.030.020.0026.475.16
160.030.020.0026.455.66
240.010.020.0026.376.47

Most erasure gains occur by γ=8\gamma=8–1212, while further scaling increases FID with limited additional suppression. We select γ=12\gamma=12 using the validation criterion detailed in the revised Appendix E.6.


[2VK4-3] (Q2) SHARE preserves unseen compositional generation capabilities.

Does preservation using a predefined retain set generalize to unseen concepts and different generation capabilities?

Can compositional generation capabilities measured by GenEval also be preserved?

SHARE preserves compositional generation performance on GenEval, whose prompts are excluded from training, Stage 2 fitting, and hyperparameter selection. For N=100N=100 celebrity erasure, SHARE scores 42.09%, close to the original model's 42.45%, and retains 96.88% of originally successful cases.

MethodGenEval (%) ↑\uparrowΔ\DeltaGenEval (pp) ↑\uparrowOriginal-success retention (%) ↑\uparrow
Original42.450.00100.00
SPEED38.87-3.5878.26
OCE35.48-6.9773.13
CPE42.05-0.4095.09
SHARE42.09-0.3696.88

We also evaluate N∈{10,50,100}N\in\{10,50,100\} in Appendix E.2.1. Across these scales, SHARE's GenEval score differs from the original by −0.36-0.36 to +0.08+0.08 percentage points, with original-success retention of 96.88%–97.77%.

These results demonstrate preservation on unseen object-compositional prompts without tailoring the retained set to GenEval, extending the preservation evidence beyond the MS-COCO distribution.

Reference

  • [Ghosh+2023] "GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment," NeurIPS 2023 Datasets and Benchmarks Track.
AuthorsOpenReview ↗

Response to Reviewer 2VK4 (2/3)

[2VK4-4] (Q3) Increasing retained-set coverage improves SHARE's image-level preservation.

Does preservation performance in generated images improve as the size of the retain set increases?

Comparing SHARE with SPEED across different retain set sizes would help clarify the practical benefits.

[TL;DR] Increasing general-retention coverage improves SHARE's retained accuracy from 92.61% to 95.46% and reduces FID from 11.97 to 3.35, while maintaining strong erasure. Under the evaluated SPEED configuration, the available null-space editing dimension collapses at coverage 0.20.2 and above. The image-level comparison is included in Appendix D.5.4.

Coverage experiment.

We vary the fraction of the full Stage 1 MS-COCO retention pool over {0,0.05,0.2,0.5,1.0}\{0,0.05,0.2,0.5,1.0\}, using the same nested retained subsets for SHARE and SPEED at each available coverage level.

Method / COCO fractionAcce↓\mathrm{Acc}_e \downarrowAccr↑\mathrm{Acc}_r \uparrowHo↑H_o \uparrowCS ↑\uparrowFID ↓\downarrow
SPEED / 0.005.4092.6493.6126.2817.92
SPEED / 0.055.8287.4790.7026.2516.57
SHARE / 0.000.8192.6195.7926.5511.97
SHARE / 0.050.2093.6596.6226.526.28
SHARE / 0.200.6194.4696.8626.574.45
SHARE / 0.500.2094.4897.0726.563.81
SHARE / 1.000.8195.4697.2826.563.35

SHARE: As coverage increases, retained accuracy improves by 2.85 points, FID decreases by 8.62, and erased accuracy remains between 0.20% and 0.81%.

SPEED: Coverage 0.050.05 slightly improves FID but reduces retained accuracy. At coverage 0.20.2 and above, the available editing subspace becomes zero-dimensional under the fixed official spectral threshold, even with IPF enabled. We report these configurations as unavailable because they cannot produce a nontrivial erasure update.

These results show SHARE's ability to exploit larger retained sets while maintaining erased-concept fitting capacity, consistent with the half-space feasible-set analysis.

The image-level sweep uses one controlled run per coverage fraction and the full Stage 1 MS-COCO pool. Figure 4 separately studies representation-level fitting and leakage with at most 2,000 retained prompts.


[2VK4-5] (Q4) SHARE supports joint multi-character erasure and composed character–style edits.

Does SHARE achieve similarly favorable trade-offs when erasing multiple concepts in other domains, such as characters or artistic styles?

We already evaluate simultaneous erasure of three characters (Snoopy, Mickey, and SpongeBob), as well as composition of this instance-erasure branch with separately trained artistic-style edits. Section 5.3, Figure 3, and Appendix E.3 (Table 13) report these evaluations using the SPEED few-concept protocol.

These experiments extend SHARE beyond celebrities to simultaneous character erasure and character–style composition. Jointly erasing multiple artistic styles within a single branch has not yet been evaluated; further multi-style experiments are in progress.


[2VK4-6] (Q5) Larger γ\gamma is needed to suppress implicit nudity concepts effectively.

Why is a substantially larger inference scaling factor used for nudity erasure compared with other concept domains?

The larger γ\gamma provides the stronger residual intervention needed for implicit nudity erasure. The sensitivity analysis in [2VK4-2] shows that γ=1\gamma=1 leaves detection rates of 0.37 on I2P, 0.39 on MMA, and 0.58 on RAB, whereas γ=12\gamma=12 reduces them to 0.03, 0.02, and 0.00, respectively.

We interpret this domain dependence through the erasure targets. Celebrity prompts explicitly name the target, and the replacement phrase "a person" is relatively close to the original representation. Nudity can be activated indirectly through visually diverse prompts, while the replacement "a person in modest clothing" involves a larger residual displacement.

The empirical sweep shows that γ=8\gamma=8–1212 achieves most of the suppression benefit; further scaling yields limited improvements while increasing FID. We therefore select γ=12\gamma=12 using validation data.

CPE (Lee+2025) similarly uses domain-dependent scaling, setting its corresponding parameter η\eta to 0.5 for celebrity erasure and 3.0 for explicit-content erasure.

Reference

  • [Lee+2025] "Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gate," ICLR 2025.
AuthorsOpenReview ↗

Response to Reviewer 2VK4 (3/3)

[2VK4-7] (Q6) Adversarial refinement and inference scaling improve robustness against UnlearnDiffAtk.

How robust is the final SHARE model against adaptive white-box attacks, such as UnlearnDiffAtk?

Adversarially refined SHARE achieves UnlearnDiffAtk ASR 0.29 at γ=24\gamma=24, with FID 6.47. CPE achieves a lower ASR of 0.21 but has substantially higher FID (20.08).

As detailed in [G-4], we evaluate UnlearnDiffAtk against the final edited checkpoints. Adversarial refinement reduces SHARE's ASR from 0.82 to 0.46 at the default γ=12\gamma=12. Increasing γ\gamma to 24 further reduces ASR to 0.29, with FID increasing from 4.82 to 6.47.

Without adversarial refinement, SHARE at γ=24\gamma=24 also matches OCE's ASR of 0.73 while achieving much lower FID (5.92 vs. 46.42).

These results demonstrate that SHARE's robustness against adaptive attacks can be strengthened through adversarial refinement and inference scaling while maintaining substantially better general-image retention than the compared methods. We have added the full comparison to Appendix D.3.


[2VK4-8] (Q7) FLUX.1 evaluation to adversarial prompts is ongoing

Could the authors extend the FLUX.1-dev evaluation beyond I2P to include adversarial benchmarks?

The revised manuscript currently reports FLUX.1-dev performance on I2P, but not MMA-Diffusion or Ring-A-Bell. We are extending the evaluation to these adversarial benchmarks using the final FLUX.1-dev checkpoint. Results are not yet available; we will report them if completed during the discussion period.

Reviewer_qhMgOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

SHARE aims to erase concepts from text-to-image models while preserving their behavior on other prompts. The starting point is that forcing retained embeddings into the null space of a linear edit can leave too little room for erasure. SHARE instead learns a shared affine selector whose nonpositive outputs are removed by a rectifier, then fits a residual readout for each layer by ridge regression. The paper derives the dimensions of the preservation-feasible sets and tests celebrity, unsafe-content, instance, style, and composed erasure on Stable Diffusion v1.4, plus an experiment on FLUX.1. Ablations examine the selector losses, retained-set size, representation dimension, and inference-time scaling.

Critical strengths

The geometric argument is clear.

The retention gains are large and supported by several measures. For 100 celebrities, Table 1 gives erased/retained accuracy of 0.81/95.46 and FID 3.35, versus 0.25/92.58 and FID 6.10 for local CPE. Table 2 shows a similar retention advantage for implicit erasure. The matched-image and projection diagnostics in Appendix E.1 support this reading of the FID results.

Critical weaknesses

The data splits and trained loss need clarification. Sections 4.3 and 5 select beta, ridge regularization, and gamma on validation data. The supplement gives celebrity configurations, but requests 29,999 COCO captions without IDs, split definitions, the validation search protocol, or evaluation scripts. This leaves generalization to unseen retained prompts and baseline tuning fairness unclear. The released code also uses ReLU(max preactivation / scale + 0.05)^2 (share/train.py, lines 92-104), whereas Eq. (44) omits the normalization and margin. That difference matters when Table 5 attributes retention partly to this loss. Main image-level results lack uncertainty across selector training seeds; Appendix D.5.1 uses one seed for the main ablation.

The benefit of Stage 2 has not been separated from that of the selector. Stage 1 already trains D to reconstruct the text-space residual (Eq. 7), so W_l D provides a direct readout. Table 5 and Figure 4 do not compare it with the ridge solution or matched jointly trained readouts. Since CPE already supplies a selective nonlinear baseline (Appendix A), this comparison would show what the analytic fitting stage adds.

Questions for authors

What are the exact training/validation/test splits, COCO caption IDs, validation search protocol, and baseline tuning budgets? Which PM loss produced the reported results, and how sensitive are they to its margin and independent training seeds?

With the same selector, how does Stage 2 compare with Delta_l = W_l D and with jointly trained readouts?

Were any evaluation attacks optimized against the final model and its selector? Appendix D.3 describes adversarial-embedding refinement, while Table 2 uses named prompt benchmarks. Would a second detector or blinded human assessment support the low detection rates?

8 comments
AuthorsOpenReview ↗

Response to Reviewer qhMg (1/3)

We thank the reviewer for the detailed and technically specific feedback. We address the shared concerns regarding data splits, independent training seeds, analytic readouts, and adaptive robustness in [G-1]–[G-4] of our General Response. Below, we provide additional details on the tuning protocol, correct the PM loss equation, and report an independent detector evaluation.


[qhMg-1] (W1, Q1) We provide exact prompt splits and hyperparameter search protocols.

The data splits and trained loss need clarification.

What are the exact training/validation/test splits, COCO caption IDs, validation search protocol, and baseline tuning budgets?

[TL;DR] The revised appendix documents the prompt splits, hyperparameter search grids, and selected configurations for SHARE's celebrity and implicit-erasure experiments. The updated supplementary material includes machine-readable prompt lists with MS-COCO caption IDs.

Prompt splits.

As detailed in [G-1], final evaluation prompts are excluded from training and hyperparameter selection. We follow the evaluation protocols of the original papers. For SHARE, training and validation prompts are constructed separately from the final evaluation prompts, including disjoint general MS-COCO captions. The corresponding prompt lists and COCO caption IDs are included in the supplementary material.

Baseline configurations.

For locally trained SPEED, OCE, and CPE baselines, we follow the training prompts and default hyperparameters from their official implementations, without additional local hyperparameter sweeps. For FLUX.1-dev, we evaluate EraseAnything using the authors' released checkpoint rather than retraining it locally. We clarify these settings in Appendix C.3 in the revision.

SHARE hyperparameter search.

We tune SHARE sequentially using held-out validation prompts:

  1. Stage 1: Search the representation dimension rr and train a selector for each candidate.
  2. Stage 2: With the selected selector fixed, tune β\beta and ρB\rho_B sequentially over the common grid {0.03,0.10,0.30,1.00}\{0.03,0.10,0.30,1.00\} for each parameter, where λΔ=ρBtr⁡(G)/r\lambda_\Delta=\rho_B\operatorname{tr}(G)/r. Each candidate requires only a closed-form readout solve.
  3. Inference: Select γ\gamma using validation prompts, without retraining the selector or readouts. We search {1.00,1.10,1.20}\{1.00,1.10,1.20\} for celebrity erasure and {4,8,12,16}\{4,8,12,16\} for implicit erasure.

The following table summarizes the searched representation dimensions and selected hyperparameters.

SettingSearched rrSelected (r,β,ρB,γ)(r,\beta,\rho_B,\gamma)
10 celebrities8, 12, 16, 24, 32, 48(24, 0.10, 0.03, 1.10)
50 celebrities32, 48, 64, 96, 128, 160(96, 0.10, 0.10, 1.10)
100 celebrities64, 128, 192, 256(192, 0.30, 0.10, 1.10)
Implicit (SD v1.4)16, 32, 64(16, 0.10, 0.03, 12)
Implicit (FLUX.1-dev)16, 32, 64, 128(128, 0.03, 0.03, 12)

For celebrity erasure, we assess erasure and retention on held-out templates using Acce\mathrm{Acc}_e, Accr\mathrm{Acc}_r, and their harmonic score HoH_o. The implicit-erasure experiments use their separate validation prompts.

The complete search grids, selected configurations, and tuning procedure are documented in Appendix C.3.

Additional held-out evaluation.

In response to Reviewer 2VK4's request ([2VK4-3]), we also evaluate the celebrity checkpoints on GenEval, whose prompts are excluded from training and hyperparameter selection.For the 100-celebrity case, SHARE achieves 42.09%, compared with 42.45% for the original model, and preserves 96.88% of originally successful cases. These results demonstrate preservation on unseen compositional prompts beyond the MS-COCO distribution. Full results are provided in Appendix E.2.1.

References

  • [Sun+2026] "Orthogonal Concept Erasure for Diffusion Models," ICML 2026.
  • [Ghosh+2023] "GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment," NeurIPS 2023 Datasets and Benchmarks.
AuthorsOpenReview ↗

Response to Reviewer qhMg (2/3)

[qhMg-2] (W1, Q1) The reported results use the normalized, margin-shifted PM loss.

Eq. (44) omits the normalization and margin.

Which PM loss produced the reported results, and how sensitive are they to its margin

[TL;DR] All submitted models use the normalized, margin-shifted PM loss implemented in the released code, with margin m=0.05m=0.05. Equation (44) omitted the normalization and margin; we have corrected it in the revision. A new margin-sensitivity analysis in Appendix E.7 confirms that the erasure-retention trade-off remains favorable across the tested margins.

Implemented loss.

The corrected objective is \mathcal{L}_{\mathrm{PM}}(\bar{\Pi}) = \sum_{g\in\mathcal{G}_P} \omega_g \mathbb{E}_{q\sim Q_R^g} \left[ \frac{1}{r} \sum_{k=1}^{r} \left( \max\left\\{ 0,\\, \max_{\ell\in[L]}\frac{[\bar{\Pi}(q_\ell)]_k }{ s_P } +m \right\\} \right)^2 \right], where sP=RP/dtexts_P=R_P/\sqrt{d_{\mathrm{text}}}, RPR_P denotes the RMS distance of retained embeddings from their center, and m=0.05m=0.05 is the default margin.

Margin sensitivity.

We retrained the 100-celebrity model with m∈{0,0.0125,0.05,0.20}m\in\{0,0.0125,0.05,0.20\}, fixing all other settings.

Margin mmAcce↓\mathrm{Acc}_e \downarrowAccr↑\mathrm{Acc}_r \uparrowHo↑H_o \uparrowFID ↓\downarrow
0.0001.8295.6396.894.45
0.01250.8194.8196.673.58
0.050 (default)0.8195.4697.283.35
0.2001.0194.8596.872.64

All tested margins maintain strong erasure and retention. The default m=0.05m=0.05 yields the highest HoH_o, while m=0.20m=0.20 achieves the lowest FID. The margin therefore controls the operating point without determining whether the method achieves a favorable trade-off.


[qhMg-3] (W1, Q1) SHARE achieves consistent results across independent training seeds.

Main image-level results lack uncertainty across selector training seeds.

How sensitive are the results to independent training seeds?

Three independently trained selectors per main setting yield consistent erasure-retention performance. As detailed in [G-2], we independently train Stage 1 with three seeds, recompute Stage 2 for each selector, and use matched evaluation conditions.

Tables 1 and 2 report means over three independent training runs, while Table 10 reports the corresponding sample standard deviations. For 100-celebrity erasure, SHARE achieves Accr=95.32±0.40\mathrm{Acc}_r=95.32\pm0.40, Ho=97.28±0.25H_o=97.28\pm0.25, and FID =3.22±0.26=3.22\pm0.26.


[qhMg-4] (W2, Q2) The selector supplies the main gain; analytic readout fitting further improves retention.

The benefit of Stage 2 has not been separated from that of the selector.

With the same selector, how does Stage 2 compare with Δℓ=WℓD\Delta_\ell=W_\ell D and with jointly trained readouts?

The learned selector accounts for most of SHARE's performance, while analytic readout fitting provides an additional retention benefit. With the same ARC selector, analytic ridge fitting improves retained accuracy by 0.85 points, HoH_o by 0.42 points, and FID by 0.30 over the direct readout Δℓ=WℓD\Delta_\ell=W_\ell D, with comparable erasure.

The matched joint optimization baseline performs substantially worse on retention: compared with joint optimization, SHARE improves retained accuracy by 3.09 points, HoH_o by 1.60 points, and FID by 1.42.

[G-3] provides the complete comparison of analytic readouts, direct decoder-based readouts, and joint optimization, including experimental controls and three-run statistics.

These results identify the learned ARC selector as the primary source of performance, while Stage 2 contributes retention-aware layer-specific fitting through a closed-form solution. The direct readout remains a competitive simpler alternative that requires no additional ridge solve.

We have clarified this distinction in Sections 4.2 and 5.4.

AuthorsOpenReview ↗

Response to Reviewer qhMg (3/3)

[qhMg-5] (Q3) We evaluate adaptive attacks against the final SHARE model.

Were any evaluation attacks optimized against the final model and its selector?

We now evaluate the final SHARE checkpoints using UnlearnDiffAtk, a white-box adversarial-prompt attack optimized against the edited model. The final model, including its ARC selector and layer-specific readouts, remains frozen during attack optimization.

As detailed in [G-4], adversarial refinement reduces UnlearnDiffAtk ASR from 0.82 to 0.46 at γ=12\gamma=12 and from 0.73 to 0.29 at γ=24\gamma=24. The refined model at γ=24\gamma=24 achieves FID 6.47, substantially below CPE's 20.08, although CPE attains a lower ASR (0.21 vs. 0.29).

This additional evaluation complements the fixed-prompt MMA-Diffusion and Ring-A-Bell benchmarks. The full results are included in the revised Appendix D.3.


[qhMg-6] (Q3) An independent VQA detector confirms substantial reductions in detected nudity.

Would a second detector or blinded human assessment support the low detection rates?

[TL;DR] An independent VQA-based nudity detector confirms substantial reductions in detected content for SHARE relative to the original model. Adversarially-refined SHARE achieves low detection rates across I2P, MMA-Diffusion, and Ring-A-Bell, consistent with the main NudeNet evaluation. The additional results and detector protocol are reported in Appendix E.2.2.

Independent detector.

Following Lin+2024 and Chen+2025, we use CLIP-FlanT5-XXL with the fixed question: ``Does this figure show nudity? Please answer yes or no.'' An image is classified as positive when the official VQAScore for the answer "Yes," including the end-of-sequence token, exceeds 0.50.5. We compute scores in BF16 with batch size one, reusing the images from the original evaluation.

Results.

The table reports VQA detection rates, with NudeNet rates in parentheses.

MethodAdv. refinementI2P ↓\downarrowMMA ↓\downarrowRAB ↓\downarrow
OriginalNo0.85 (0.83)0.89 (0.60)1.00 (1.00)
SPEEDNo0.22 (0.20)0.43 (0.23)0.95 (0.20)
OCENo0.18 (0.11)0.24 (0.10)0.72 (0.08)
SHARENo0.13 (0.12)0.20 (0.12)0.02 (0.00)
CPEYes0.04 (0.03)0.03 (0.02)0.02 (0.00)
SHAREYes0.04 (0.03)0.04 (0.02)0.01 (0.00)

The VQA detector is generally more sensitive than NudeNet, particularly on MMA and RAB. Despite detector-dependent differences, both evaluations show substantial reductions for SHARE relative to the original model.

With adversarial refinement, SHARE achieves VQA detection rates of 0.04 on I2P, 0.04 on MMA, and 0.01 on RAB, comparable to CPE. The nonzero RAB detections also reveal residual unsafe content missed by NudeNet.

We have added the detector protocol and per-benchmark results to Appendix E.2.2 and revised the corresponding claims to reflect these findings.

References

  • [Lin+2024] "Evaluating Text-to-Visual Generation with Image-to-Text Generation," ECCV 2024.
  • [Chen+2025] "Comprehensive Assessment and Analysis for NSFW Content Erasure in Text-to-Image Diffusion Models," NeurIPS 2025 Datasets and Benchmarks Track.
Reviewer_JY2ZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies the trade-off between concept erasure and non-target preservation in text-to-image models. It proposes replacing strict zero-projection constraints with rectified inactivity: retained representations may have non-positive preactivations while still producing zero residual updates after ReLU. SHARE implements this idea in two stages, learning a shared affine selector through residual reconstruction and preservation objectives, then computing layer-specific residual readouts via closed-form ridge regression. The paper provides a geometric analysis of the preservation-feasible parameter space and evaluates the method across concept types, erasure scales, prompt conditions, and model architectures. Ablations and preservation diagnostics further examine its design and behavior.

Critical strengths
  1. The paper is well motivated and presents a coherent method. It addresses the tension between concept erasure and non-target preservation by replacing strict zero-projection requirements with rectified inactivity, offering an intuitive and mathematically grounded approach. Combining a shared selector with layer-specific analytic readouts turns this idea into a concrete editing procedure.

  2. The experiments demonstrate a competitive erasure–preservation trade-off across multiple settings. SHARE effectively suppresses target concepts while maintaining non-target recognition and limiting changes to general generation behavior. Evaluations across concept types and model architectures, together with component ablations and representation- and image-level diagnostics, provide complementary support for the overall method’s effectiveness.

Critical weaknesses
  1. The methodological novelty is incremental, and the theory provides only a partial explanation of the empirical gains. SHARE combines established nonlinear gating and analytic regression with a selector shared across target layers through a rectified half-space formulation and a two-stage procedure. Although the combination performs well, its independent contribution over earlier nonlinear or gated editing methods is not yet sufficiently established. The feasible-dimension analysis provides useful geometric motivation by showing that half-space preservation can enlarge the preservation-feasible representation space. However, it does not characterize when erased concepts will activate useful coordinates or when the resulting representation will support accurate recovery of the desired residuals; these properties are instead encouraged through the learned Stage-1 objectives. Thus, the theory only partially explains the observed erasure–preservation improvement.

  2. The ablations do not sufficiently isolate the benefits of the central design choices. The null-space and half-space alternatives differ in how their representations are constructed and whether those representations are learned using task supervision. Their performance difference therefore does not isolate the effect of relaxing equality-based preservation. The loss ablations establish that the training objectives matter, but they do not resolve this attribution issue. Similarly, there is no direct comparison between analytic readouts and joint optimization under the same residual architecture. This leaves the independent value of the two-stage decomposition insufficiently demonstrated.

Questions for authors
  1. Beyond the existing loss ablations, what evidence separates the benefit of half-space preservation from that of learning a task-specific nonlinear representation? Clarifying this would help establish the contribution beyond existing gated editing methods.

  2. What empirical advantage does separating selector learning from analytic readout fitting provide over joint optimization under the same residual architecture, particularly in terms of the erasure–preservation trade-off?

5 comments
AuthorsOpenReview ↗

Response to Reviewer JY2Z (1/2)

We thank the reviewer for the thoughtful assessment and for identifying the need to separate the contributions of preservation geometry, task-specific representation learning, and readout fitting. We clarify the scope of our theoretical claims and provide matched ablations to better isolate the central design choices. The readout comparisons shared with Reviewer qhMg are summarized in [G-3] of our General Response.


[JY2Z-1] (W1) SHARE combines rectified half-space preservation with residual-aligned representation learning.

The methodological novelty is incremental, and the theory provides only a partial explanation of the empirical gains.

SHARE's central contribution is a learned rectified half-space selector that preserves erasure-relevant information while suppressing retained-concept activations. Proposition 1 establishes that half-space preservation admits more feasible representation directions than equality-based preservation under Assumption 1. The residual-aligned Stage 1 objectives then learn directions that support erased-concept residual recovery.

These components have complementary roles: the geometric analysis characterizes preservation feasibility, while Stage 1 learns an effective representation within the relaxed space.

Compared with existing nonlinear editing methods such as CPE (Lee+2025) and ETC (Seo+2026), SHARE combines a shared residual-supervised selector with closed-form layer-specific readouts, as discussed in Section 4.4 and Appendix A.

We further isolate the role of preservation geometry through matched learned equality-based baselines in the revised Figure 4 (Section 5.4 and Appendix D.5.3). These comparisons directly test whether task-specific learning alone explains the observed performance, as detailed in our reply below.

AuthorsOpenReview ↗

Response to Reviewer JY2Z (2/2)

[JY2Z-2] (W2, Q1) Half-space preservation improves the fitting-preservation trade-off beyond task-specific learning.

The ablations do not sufficiently isolate the benefits of the central design choices.

What evidence separates the benefit of half-space preservation from that of learning a task-specific nonlinear representation?

[TL;DR] Half-space preservation maintains low retained leakage and accurate erased-residual fitting as retained-set coverage increases, even compared with task-supervised null-space alternatives. The revised Figure 4 isolates this benefit using learned equality-based controls matched in supervision, representation dimension, optimization budget, and readout fitting.

Controlled comparison.

The revised Figure 4 compares five variants:

  1. Closed-form null space: A null-space representation constructed without task-specific selector learning.
  2. Learned null space (new): A task-supervised selector trained with a soft equality penalty.
  3. Projected learned null space (new): The task-supervised selector projected onto the SPEED-style null space.
  4. Linear half space: A learned half-space selector without an affine offset.
  5. Affine half space: SHARE's learned affine half-space selector.

All the four learned variants use the same training data, representation dimension, erased-residual targets, optimization budget, and readout-fitting protocol.

Results.

Task-specific learning alone improves residual fitting but does not reproduce the half-space variants' fitting-preservation trade-off:

  • Learned null space achieves lower fitting error than Closed-form null space from coverage 0.20.2 onward, but incurs substantially higher retained leakage.
  • Projected learned null space reduces leakage but has higher fitting error than Closed-form null space at every coverage level.
  • Linear and Affine half space maintain substantially lower leakage as coverage increases. Affine half space keeps fitting error nearly constant and achieves the lowest fitting error from coverage 0.20.2 onward. At coverage 0.050.05, Closed-form null space has slightly lower fitting error, while the half-space variants achieve lower leakage.

Thus, the half-space formulation preserves erasure-relevant fitting capacity while suppressing retained responses under increasing retention coverage. This empirical advantage is consistent with the enlargement of preservation-feasible directions established by Proposition 1.

We have added these matched comparisons to Section 5.4, Figure 4, and Appendix D.5.3.


[JY2Z-3] (W2, Q2) Two-stage training improves retention over joint optimization at comparable erasure strength.

there is no direct comparison between analytic readouts and joint optimization under the same residual architecture.

What empirical advantage does separating selector learning from analytic readout fitting provide over joint optimization?

Two-stage training achieves substantially better retention than joint optimization while maintaining comparable erasure strength. In 100-celebrity erasure, it improves retained accuracy by 3.09 points, HoH_o by 1.60 points, and FID by 1.42.

As detailed in [G-3], we compare the proposed method against joint optimization of the selector and all layer-specific readouts under matched architecture, representation dimension, training data, edited layers, optimization update number, and evaluation conditions. Results averaged over three independent training runs are:

MethodAcce↓\mathrm{Acc}_e \downarrowAccr↑\mathrm{Acc}_r \uparrowHo↑H_o \uparrowFID ↓\downarrow
Two-stage learning0.68±0.230.68 \pm 0.2395.32±0.4095.32 \pm 0.4097.28±0.2597.28 \pm 0.253.22±0.263.22 \pm 0.26
Joint optimization0.71±0.180.71 \pm 0.1892.23±0.5192.23 \pm 0.5195.68±0.3295.68 \pm 0.324.64±0.414.64 \pm 0.41

The advantage comes from the two-stage design: Stage 1 learns a shared representation aligned with the text-space erasure residual, and Stage 2 fits retention-aware layer-specific readouts to that fixed representation.

The revised Section 5.4 includes this comparison alongside the direct decoder-based readout Δℓ=WℓD\Delta_\ell=W_\ell D. Together, these experiments establish the practical benefit of separating selector learning from readout fitting.

AuthorsOpenReview ↗

Gentle Follow Up

Dear Reviewer JY2Z,

Thank you again for your thoughtful and constructive feedback.

As the discussion period approaches its end, we would appreciate any feedback on whether our responses and the revised manuscript have adequately addressed your concerns.

Please let us know if any questions or concerns remain. We would be happy to provide further clarification.

Best regards,

The Authors

Reviewer_m1UzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work tackles concept erasure in text-to-image diffusion models. SHARE replaces exact linear preservation constraints with a rectified half-space selector, then pairs a learned shared selector with closed-form residual readouts for individual layers. The experiments span multi-concept, implicit, adversarial, instance, and style erasure on Stable Diffusion and FLUX.1.

Critical strengths

A clear geometric result. Appendix B.1 characterizes the feasible-dimension gap as (r(s-\ell)), where (s) is the retained embedding span dimension and (\ell) is the retained cone's lineality dimension. This tells us when half-space preservation adds feasible directions. Promising results across several settings. In Table 1, SHARE's harmonic scores for 50 and 100 erased celebrities are 98.20 and 97.28, respectively, compared with 96.42 and 96.03 for locally evaluated CPE. Table 2 covers implicit and adversarial prompts, and Table 4 tests FLUX.1. Detailed method and limitations. Appendix C lays out the objectives, embedding groups, weighting, and training algorithms. Section 6 discusses retraining requirements and notes that the method has not been tested at the thousand-concept scale.

Critical weaknesses

Training, tuning, and evaluation prompt splits are not specified clearly. The paper names the prompt sources: Stage 1 uses retained-celebrity and general MS-COCO prompts, while evaluation uses five benchmark templates and the first 1K MS-COCO prompts. It also says that (\beta), (\lambda_\Delta), and (\gamma) are selected on validation data. What is missing is the prompt-instance split across training, validation, and final evaluation, including whether these sets are disjoint. Without that information, readers cannot verify that the reported results are based on held-out prompts. The main results lack run-to-run uncertainty. Appendix D.5.1 explicitly says that the full SHARE (N=100) checkpoint uses one training seed. Appendix D.5.2 reports five-seed means for a retained-set-coverage sweep, but its full-coverage point reuses that checkpoint without retraining. The paper does not report variability across independently trained models for the main comparisons. I would not infer that every model in every experiment used only one seed; the explicit statement is about the full (N=100) checkpoint. Still, the reported results do not show how stable the comparisons are across training runs, especially where margins over local baselines are small. The practical selector is not shown to satisfy the conditions of the exact preservation result. Proposition 1 establishes a geometric enlargement of the feasible set when the stated half-space constraints hold. Section 4.2 trains the selector with losses that penalize retained responses, rather than enforcing those inequalities as hard constraints. The paper reports retention metrics but not how often retained-prompt preactivations remain positive after training, it limits how directly the geometric result supports exact preservation by the learned model.

Questions for authors

Are the prompts used to train the selector, choose parameters on validation data, and run the final evaluation disjoint? Please provide the prompt-level splits. Can you report variability across independently trained SHARE models for the main comparisons? Across the prompts used for evaluation, how often do retained-prompt preactivations remain positive after training, and how does this relate to the reported retention metrics?

2 comments
AuthorsOpenReview ↗

Response to Reviewer m1Uz

We thank the reviewer for the careful and constructive assessment. We especially appreciate the recognition of our geometric analysis and the breadth of the evaluation. We address the shared concerns regarding data splits and training variability in [G-1] and [G-2] of our General Response. Below, we also provide new analyses of retained-prompt activations.


[m1Uz-1] (W1, Q1) Training, validation, and evaluation prompts are disjoint.

Training, tuning, and evaluation prompt splits are not specified clearly.

Are the prompts used to train the selector, choose parameters on validation data, and run the final evaluation disjoint? Please provide the prompt-level splits.

Final evaluation prompts are excluded from both training and hyperparameter selection. As detailed in [G-1], the training, validation, and evaluation templates are disjoint. General MS-COCO training and evaluation captions are also disjoint. We have documented the complete splits in the revised appendix and included machine-readable prompt lists in the updated supplementary material.


[m1Uz-2] (W2, Q2) SHARE achieves consistent performance across independent training runs.

The main results lack run-to-run uncertainty.

Can you report variability across independently trained SHARE models for the main comparisons?

Three independently trained SHARE models per main setting show small performance variation and consistent erasure-retention results. As detailed in [G-2], we independently train Stage 1 with three seeds and recompute Stage 2 for each selector.

Tables 1 and 2 report means over three independent training runs, while Table 10 reports the corresponding sample standard deviations. For 100-celebrity erasure, SHARE achieves Ho=97.28±0.25H_o=97.28\pm0.25 and FID =3.22±0.26=3.22\pm0.26, maintaining its advantage over the locally evaluated baselines.


[m1Uz-3] (W3, Q3) The learned selector strongly suppresses retained-prompt activations.

The practical selector is not shown to satisfy the conditions of the exact preservation result.

Across the prompts used for evaluation, how often do retained-prompt preactivations remain positive after training, and how does this relate to the reported retention metrics?

[TL;DR] The learned selector substantially suppresses positive preactivations on retained prompts: the rates are 17.11% for retained celebrities and 0.52% for general MS-COCO prompts, compared with 62.20% for erased celebrities. Despite nonzero retained activations, SHARE achieves 95.32±0.4095.32\pm0.40 retained accuracy in the 100-celebrity setting. Figure 9 in the revision further shows that smaller selector activations are associated with smaller projection-output and image-level deviations.

Connection to Proposition 1.

Proposition 1 characterizes the feasible geometry under the exact homogeneous constraint ΠCR≤0\Pi C_R\leq0. SHARE operationalizes this principle using an affine selector and soft penalties, which encourage retained-prompt inactivity without enforcing exact nonpositivity.

We now distinguish these theoretical and practical settings explicitly. The following measurements quantify how closely the learned selector approaches retained-prompt inactivity.

Positive-preactivation frequency.

For each evaluation prompt, we compute Πˉ(qℓ)=E(qℓ−μ)+b\bar{\Pi}(q_\ell)=E(q_\ell-\mu)+b and measure the fraction of token-coordinate pairs with [Πˉ(qℓ)]k>0[\bar{\Pi}(q_\ell)]_k>0.

Prompt groupPositive-preactivation rate
Erased celebrities62.20%
Retained celebrities17.11%
General retained prompts (MS-COCO)0.52%

Positive preactivations remain on some retained inputs, but occur 45.09 percentage points less frequently for retained celebrities than for erased celebrities.

Prompt-level activation magnitude.

We measure the rectified sequence-level activation ∥σ(Πˉ(q))∥F\|\sigma(\bar{\Pi}(q))\|_F.

Prompt groupMedian ∥σ(Πˉ(q))∥F\|\sigma(\bar{\Pi}(q))\|_F
Erased celebrities90.19
Retained celebrities15.37
General retained prompts (MS-COCO)0.21

Median activation is approximately 5.9 times smaller for retained celebrities and 429 times smaller for general retained prompts than for erased celebrities.

Relationship to retention.

Figure 9 in the revised Appendix E.2.1 plots ∥σ(Πˉ(q))∥F\|\sigma(\bar{\Pi}(q))\|_F against (1) normalized deviations in edited cross-attention projection outputs and (2) image-level deviations from the original model, using matched prompts, generation seeds, and initial latents.

Retained prompts concentrate in the low-activation, low-deviation region, while erased prompts generally exhibit larger activations and deviations. These results connect the learned selector's activation behavior to SHARE's preservation performance at both representation and image levels.

AuthorsOpenReview ↗

Gentle Follow Up

Dear Reviewer m1Uz,

Thank you again for your thoughtful and constructive feedback.

As the discussion period approaches its end, we would appreciate any feedback on whether our responses and the revised manuscript have adequately addressed your concerns.

Please let us know if any questions or concerns remain. We would be happy to provide further clarification.

Best regards,

The Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response (1/2)

We appreciate all reviewers' thoughtful and constructive feedback. We address four concerns raised by multiple reviewers in this general response:

  • [G-1] Training / Validation / Evaluation Data Splits (Reviewers m1Uz, qhMg, and 2VK4)
  • [G-2] Variability Across Independent Training Seeds (Reviewers m1Uz and qhMg)
  • [G-3] Contribution of Stage 2 / Analytic Readouts (Reviewers JY2Z and qhMg)
  • [G-4] Robustness Against Adaptive White-Box Attacks (Reviewers qhMg and 2VK4)

Reviewer-specific questions and additional analyses are addressed in our individual responses. We have also updated the manuscript and supplementary material accordingly.


[G-1] Training, validation, and evaluation prompts are disjoint.

[TL;DR] SHARE uses disjoint training, validation, and final evaluation prompts. We follow the original evaluation protocols and use official repository defaults for locally trained baselines. The revised Appendices D.2 and D.3 document the split definitions for our main experiments, and the updated supplementary material provides machine-readable prompt lists in prompts/.

Below, we describe the celebrity-erasure case as an example.

General MS-COCO prompts in celebrity erasure.

The final evaluation uses the first 1,000 captions from coco_30k_val.csv in the official OCE repository (Sun+2026). We construct the Stage 1 general-retention pool from coco_30k.csv by excluding these captions and verify that no exact caption overlap remains. MS-COCO prompts are not used for hyperparameter validation. The lists are provided in prompts/celebrity/general_train.txt and prompts/celebrity/general_eval.txt.

Erased- and retained-celebrity templates.

Following OCE (Sun+2026) and related work (Lee+2025, Seo+2026), we use disjoint template sets for the three stages:

  • Training: 500 templates.
  • Validation: 20 templates, applied to both erased and retained identities for hyperparameter selection using Acce\mathrm{Acc}_e and Accr\mathrm{Acc}_r.
  • Evaluation: Five fixed OCE templates, excluded from training and validation.

The exact templates are provided in prompts/celebrity/template_train.txt, prompts/celebrity/template_val.txt, and prompts/celebrity/template_eval.txt. The revised appendix also details the validation protocol and experimental configurations.

References

  • [Lee+2025] "Concept Pinpoint Eraser for Text-to-image Diffusion Models via Residual Attention Gate," ICLR 2025.
  • [Sun+2026] "Orthogonal Concept Erasure for Diffusion Models," ICML 2026.
  • [Seo+2026] "Erasing Thousands of Concepts: Towards Scalable and Practical Concept Erasure for Text-to-Image Diffusion Models," CVPR 2026.

[G-2] SHARE achieves consistent results across independent training runs.

[TL;DR] Three independently trained SHARE models per main setting show low run-to-run variability and consistent erasure-retention performance. Tables 1 and 2 report means over three independent training runs, while Table 10 reports the corresponding sample standard deviations.

Experimental protocol.

We independently train Stage 1 with three seeds and recompute Stage 2 for each selector, keeping data splits and hyperparameters fixed. We also fix evaluation prompts, generation seeds, initial latents, and sampling settings to isolate variability from selector training.

This protocol covers all celebrity-erasure scales in Table 1 and implicit-erasure settings in Table 2.

Results for 100-celebrity erasure.

SHARE results are means and sample standard deviations across three independently trained models. Baseline results are from the locally evaluated single models in the submission.

MethodAcce↓\mathrm{Acc}_e \downarrowAccr↑\mathrm{Acc}_r \uparrowHo↑H_o \uparrowCS ↑\uparrowFID ↓\downarrow
SPEED (local)5.4092.6493.6126.2817.92
OCE (local)4.5192.4093.9225.9217.75
CPE (local)0.2592.5896.0326.546.10
SHARE (3 runs)0.68±0.230.68 \pm 0.2395.32±0.4095.32 \pm 0.4097.28±0.2597.28 \pm 0.2526.58±0.0126.58 \pm 0.013.22±0.263.22 \pm 0.26

SHARE's standard deviations are small relative to its performance differences from SPEED and OCE. Compared with CPE, SHARE achieves higher retained accuracy (95.32 vs. 92.58) and lower FID (3.22 vs. 6.10), while CPE achieves slightly lower erased accuracy (0.25 vs. 0.68).

AuthorsOpenReview ↗

General Response (2/2)

[G-3] The learned selector provides the main gain, complemented by analytic readout fitting.

[TL;DR] The learned ARC selector accounts for most of SHARE's performance, while analytic ridge readouts further improve retention. Two-stage training outperforms matched joint optimization at comparable erasure strength (Section 5.4).

Results for 100-celebrity erasure (mean ±\pm sample SD, three independent runs).

MethodAcce↓\mathrm{Acc}_e \downarrowAccr↑\mathrm{Acc}_r \uparrowHo↑H_o \uparrowFID ↓\downarrow
Analytic readout with ARC (SHARE)0.68±0.230.68 \pm 0.2395.32±0.4095.32 \pm 0.4097.28±0.2597.28 \pm 0.253.22±0.263.22 \pm 0.26
Direct readout with ARC0.61±0.250.61 \pm 0.2594.47±0.3894.47 \pm 0.3896.86±0.2596.86 \pm 0.253.52±0.513.52 \pm 0.51
Joint optimization0.71±0.180.71 \pm 0.1892.23±0.5192.23 \pm 0.5195.68±0.3295.68 \pm 0.324.64±0.414.64 \pm 0.41

Benefit of analytic readout fitting.

With the same ARC selector and decoder, analytic ridge fitting improves retained accuracy by 0.85 points, HoH_o by 0.42 points, and FID by 0.30 over the direct readout Δℓ=WℓD\Delta_\ell=W_\ell D, at comparable erasure. The competitive direct readout shows that the selector provides most of the gain; analytic fitting adds closed-form, retention-aware, layer-specific readouts.

Benefit of two-stage training.

At comparable erased accuracy, two-stage training improves retained accuracy by 3.09 points, HoH_o by 1.60 points, and FID by 1.42 over joint optimization. Thus, learning a residual-aligned shared representation before fitting layer-specific readouts improves retention under the evaluated protocol.

Matched experimental design.

We compare three configurations:

  1. Analytic readout with ARC (SHARE): Train the shared selector and auxiliary decoder DD in Stage 1, freeze the selector, and compute retention-aware ridge readouts in Stage 2.
  2. Direct readout with ARC: Use the same Stage 1 selector and decoder, replacing ridge fitting with Δℓ=WℓD\Delta_\ell=W_\ell D.
  3. Joint optimization: Remove DD and jointly optimize the selector Π\Pi and layer-specific readouts Δℓ\Delta_\ell using the corresponding erased- and retained-residual objectives.

All variants match the residual architecture, dimension, training data, edited layers, optimization update count, evaluation prompts, generation seeds, and initial latents. The first two share a trained selector; joint optimization learns its own selector and readouts.

These comparisons identify SHARE's learned rectified half-space selector as its central contribution and quantify the added retention benefit of analytic readouts and two-stage training.


[G-4] Adaptive white-box evaluation reveals a robustness–retention trade-off.

[TL;DR] We evaluated the final SHARE checkpoints against UnlearnDiffAtk (Zhang+2024), an adaptive white-box adversarial-prompt attack. Adversarial refinement reduces SHARE's attack success rate (ASR) from 0.82 to 0.46 at γ=12\gamma=12 and from 0.73 to 0.29 at γ=24\gamma=24. At γ=24\gamma=24, refined SHARE achieves ASR 0.29 with FID 6.47, compared with CPE's ASR 0.21 and FID 20.08. We have added this evaluation to Appendix D.3.

Evaluation protocol.

Unlike the fixed-prompt MMA-Diffusion and Ring-A-Bell benchmarks (Table 2), UnlearnDiffAtk optimizes adversarial prompts against each frozen final edited model under a white-box threat model. We follow the official implementation's main attack settings, evaluating SHARE with and without adversarial refinement at γ∈{12,24}\gamma\in\{12,24\} alongside SPEED, OCE, and CPE.

For SHARE, attack gradients pass through the complete edited model, including its ARC selector and layer-specific readouts. Given the attack's computational cost, we evaluate one trained checkpoint per refinement setting; FID values thus differ from the three-run means in Table 2. We report UnlearnDiffAtk ASR (lower is better) and FID on general MS-COCO prompts (lower indicates better non-target retention).

Results.

MethodAdv.UnlearnDiffAtk ASR ↓\downarrowFID ↓\downarrow
SPEEDNo0.8939.64
OCENo0.7346.42
SHARE (γ=12\gamma=12)No0.824.05
SHARE (γ=24\gamma=24)No0.735.92
CPEYes0.2120.08
SHARE (γ=12\gamma=12)Yes0.464.82
SHARE (γ=24\gamma=24)Yes0.296.47

Effect of adversarial refinement and inference scaling.

Refinement reduces ASR at both scales. Increasing γ\gamma further lowers ASR from 0.82 to 0.73 without refinement and 0.46 to 0.29 with refinement, at higher FID. Without refinement, SHARE (γ=24\gamma=24) matches OCE's ASR 0.73 with much lower FID (5.92 vs. 46.42). With refinement, CPE has lower ASR (0.21 vs. 0.29), but SHARE (γ=24\gamma=24) has much lower FID (6.47 vs. 20.08).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.