acceptodds
Under review as a conference paper at ICLR 2027

TILDE: TILt-based Distributional Erasure for Concept Unlearning

Abstract

Concept unlearning in text-to-image diffusion and rectified-flow models is critical for safe and practical deployment: with rising privacy concerns, copyright disputes, trademark constraints, and safety regulations, deployed systems must reliably suppress unwanted concepts after training. Existing methods often remove the target concept effectively, but practical unlearning also requires an equally fundamental property: the unlearned model should retain quality, diversity, and semantic coverage on benign generation. The gold standard is a retain-only model trained from scratch without the unwanted data. However, common erasure objectives do not specify which post-unlearning distribution should approximate this reference, leaving retention as an implicit byproduct of heuristic updates. We propose TILDE, TILt-based Distributional Erasure, which formulates concept unlearning as a distributional alignment problem: the desired target is the minimum-deviation conditional distribution from the pretrained model under a forgetting constraint. This energy-tilted, anchor-free target downweights concept-expressing images while preserving benign relative mass for each prompt. We approximate this target with residual -GFlowNet training for diffusion and rectified-flow generators, learning only a score correction relative to the pretrained sampler via a modular, differentiable concept energy. Across objects, artistic styles, characters and NSFW content, TILDE achieves strong forgetting while improving retention and distributional fidelity over prior baselines.

Primary area:
generative models
Keywords:
Diffusion Models, Machine Unlearning, Text-to-Image Generation

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_MY97OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • This paper investigates the idea of machine unlearning in terms of tilted distributional erasure. Based on the idea of ∇-GFlowNet, this paper formalizes unlearning in terms of KL minimization and a penalty function for unwanted data generation. The paper includes various experimental scenarios of unlearning.
Critical strengths
  • The paper investigates an important question in the unlearning domain.
  • The paper presents well-supported training strategies by defining unlearning in terms of KL divergence.
  • The paper includes various experiments to back up the proposed method.
Critical weaknesses

Please refer to the Questions below.

Questions for authors
  • Using residual score or one-step reward gradient is already proposed in ∇-GFlowNet as the author mentioned. Thus, I believe that the novelty of this paper is marginal. What is the main difference compared to ∇-GFlowNet in the methodology?

  • The right semantic example of Figure 1 shows the optimal results of the proposed method against other methods. Can we observe the similar outputs in toy or real data?

  • Can the authors explain in detail why preserving the ratio of benign data is optimal for machine unlearning? If the training data is changed due to the deletion of forget data, it is reasonable that the final output distribution is not akin to the total distribution of the original model with full data even if we delete the forget region in the distributional sense. If we consider the understanding of optimal transport, it might be optimal to transport all the mass of forget data into the near mode.

  • If we choose KL minimization as a criterion, I agree that an energy-tilted distribution can be utilized. However, there are many methods that implement KL minimization, e.g., KL-regularized reward optimization. The authors compare against DDPO in Table 3, but it does not clarify whether DDPO uses the same KL-regularized objective, making it difficult to determine whether the improvement comes from the objective choice or the optimization.

  • The computational analysis against the existing method is not well provided, which is one of the main problem of energy-based models.

  • Dealing the normalization constant in the energy-based model is very important. The authors avoid explicitly computing the normalization constant by matching gradients. However, how do the approximations in this procedure affect the resulting distribution, especially in unlearning?

2 comments

Q1. Methodological difference

We adopt the residual score-matching idea from ∇\nabla-GFlowNet because it provides a formulation for approximating energy-tilted distributions in latent space; we do not claim ∇\nabla-GFlowNet itself as our contribution. Rather, our core contributions lie in formulating concept unlearning as an anchor-free distributional alignment problem and adapting ∇\nabla-GFlowNet with critical unlearning-specific mechanisms:

  1. Posing concept unlearning as constrained distributional alignment (Proposition 1): Whereas ∇\nabla-GFlowNet was designed for proportional sampling from unnormalized reward densities, concept unlearning requires suppressing a specific target while preserving benign generation. We formalize this via constrained KL minimization (Eq. 1), deriving the prompt-conditioned Gibbs target.

  2. Adapting ∇\nabla-GFlowNet with a thresholded forget energy & multi-architecture/multi-scorer extensions: Directly applying ∇\nabla-GFlowNet with a continuous penalty destroys benign retention (Figure 3). We introduce a thresholded forget energy with a zero-penalty region and extend the stochastic transitions to Rectified-Flow transformers (SD3, FLUX) and VLM/classifier scorers.

  3. Proposing Anchor-Free Unlearning: Several prior methods map forget concepts to fixed anchors, while recent literature (AGE (ICLR 2025), LACU) motivates adaptive anchors to improve preservation. TILDE instead defines a target without prescribing a replacement anchor. Our experiments show strong retention and robustness to relearning, although they do not isolate anchor removal as the cause of these gains.

Q2. Controlled evidence for the distributional illustration

Thank you for raising this point. Yes. We evaluated TILDE on a controlled 2D mixture of eight overlapping Gaussian components (using exact mixture reverse transitions and a compatible Gaussian prior) where one target component is suppressed by ρ=0.5\rho=0.5 (moderate) or ρ=0.1\rho=0.1 (strong), trained for 30,000 steps across 5 fresh seeds per strength and evaluated on 1,000,000 samples per seed. Across both strengths, TILDE suppresses the target mode while preserving the relative proportions of all seven benign modes, reducing the 2D Jensen-Shannon (JS) divergence from 0.0053→0.001090.0053 \to \mathbf{0.00109} for ρ=0.5\rho=0.5 (exact-sampler floor: 0.001070.00107) and from 0.0230→0.001100.0230 \to \mathbf{0.00110} for ρ=0.1\rho=0.1 (exact-sampler floor: 0.001070.00107). We have included the full experimental setup, quantitative table, 2D endpoint density heatmaps, and unwrapped angular marginal curves in Appendix F (Table 6 and Figures 7 and 8) of the revised manuscript.

Q3. Distributional preservation, retraining, and the connection to FADE

TL;DR: Without access to an infeasible retain-only retrained model, the Gibbs tilted distribution pC∗(x∣y)p_{\mathcal{C}}^*(x\mid y) is the minimum-KL deviation from ppre(x∣y)p_{\mathrm{pre}}(x\mid y) that proportionally reweights zero-energy benign outputs without altering their relative probabilities; empirically, TILDE also achieves the best FADE distributional alignment scores (Table 1).

Ideally, an unlearned model should match a retain-only model pretain(x∣y)p_{\mathrm{retain}}(x\mid y) retrained from scratch, which is computationally infeasible when it comes to large models. As discussed in FADE (Cho et al., 2025), evaluating distributional alignment (DKL(pretainy∥punlearny)+DKL(punlearny∥pretainy)D_{\mathrm{KL}}(p_{\mathrm{retain}}^y\\\|p_{\mathrm{unlearn}}^y)+D_{\mathrm{KL}}(p_{\mathrm{unlearn}}^y\\\|p_{\mathrm{retain}}^y)) is essential because point-wise erasure metrics hide distributional drift. Without access to pretainp_{\mathrm{retain}}, under the stated regularity conditions, the minimum-deviation distribution satisfying the forget constraint relative to ppre(x∣y)p_{\mathrm{pre}}(x\mid y) is the tilted distribution pC∗(x∣y)=ppre(x∣y)exp⁡(−βEC(x))/Zβ,C(y)p_{\mathcal{C}}^*(x\mid y)=p_{\mathrm{pre}}(x\mid y)\exp(-\beta E_{\mathcal{C}}(x))/Z_{\beta,\mathcal{C}}(y). Suppressing the target region (E~C(x)>0\tilde{E}_{\mathcal{C}}(x)>0) and renormalizing by 1/Zβ,C(y)1/Z_{\beta,\mathcal{C}}(y) multiplies the densities of zero-energy benign outputs (E~C(x)=0\tilde{E}_{\mathcal{C}}(x)=0) by the same factor, preserving their pretrained conditional relative probabilities. This is proportional reweighting, not a nearest-mode transport rule; context preservation depends on what ppre(x∣y)p_{\mathrm{pre}}(x\mid y) supports. Fixed-anchor methods instead prescribe a replacement concept, which can introduce collateral changes; anchor choice alone does not establish whether revival occurs. We also empirically evaluate distributional agreement using FADE rather than relying only on the analytical argument. The results show that TILDE achieves the best scores in our table while achieving the strong unlearning performance.

Q4. KL-regularized reward optimization and the role of GFlowNet

TL;DR: While output-level KL-regularized RL shares the same theoretical Gibbs optimum, policy-gradient methods (DDPO/DPOK) rely on scalar trajectory returns and struggle with the forgetting-retention trade-off, whereas TILDE leverages first-order differentiable energy gradients and step-wise residual balance.

Thank you for pointing this out. We agree that it is important to separate the distributional objective from the optimization method used to learn it. With the tilt strength absorbed into E~C\widetilde E_{\mathcal C}, as in the paper, KL-regularized reward maximization

-D_{\mathrm{KL}}\\!\left(p(\cdot\mid y)\\,\\\|\\,p_{\mathrm{pre}}(\cdot\mid y)\right)$$ has the same Gibbs optimum $p^*(x\mid y)\propto p_{\mathrm{pre}}(x\mid y)e^{-\widetilde E_{\mathcal C}(x)}$ as TILDE’s practical target (Eq. 4). This equivalence is at the output-distribution level. DPOK uses trajectory KL, which upper-bounds output KL and also constrains denoising paths; with a fixed prior and restricted transitions, its optimum need not have the same output marginal. Using the same TILDE setup, we evaluated both: - **DDPO:** Maximizes the terminal forgetting reward via clipped importance-sampling policy-gradient updates over denoising steps, without an explicit KL penalty to the frozen pretrained model. - **DPOK:** Uses the same terminal forgetting reward with an explicit trajectory-level KL penalty against the frozen pretrained model. As shown in Table 3, both DDPO and DPOK achieve strong forgetting, with unlearning accuracies of **0.97** and **0.99**, respectively. However, their related/general retention scores are **0.45/0.82** for DDPO and **0.59/0.79** for DPOK. TILDE achieves **0.99** unlearning accuracy while maintaining higher related and general retention of **0.76** and **0.88**, respectively. These results show that TILDE provides a better balance between forgetting and preservation in this comparison. **Residual $\nabla$-GFlowNet** offers two relevant mechanisms that may contribute to this performance: 1. **First-Order Differentiable Energy Gradient.** The reward terms in DDPO and DPOK use score-function policy gradients weighted by scalar returns, without differentiating the forget energy with respect to the generated image. In contrast, TILDE uses the energy gradient through the predicted clean image, $-\bar\alpha_t\nabla_{z_t}\widetilde E_{\mathcal C}(\hat x_\theta)$, together with the learned continuation residual $g_\phi(z_t,t)$ (Eq. 8), supplying an explicit local direction for reducing concept evidence during denoising. 2. **Step-Wise Residual Detailed Balance.** The forward and reverse losses encourage step-wise consistency between the local transition-score correction and the continuation score relative to $p_{\mathrm{pre}}$ (Eqs. 6 to 9). Below the threshold $\tau$, the direct energy-gradient term vanishes identically, avoiding a direct forgetting force on low-evidence estimates. The continuation term $g_\phi$ and shared parameter updates can still change these transitions, so this property alone does not guarantee benign preservation. These design differences may contribute to the balance between forgetting and preservation observed in Table 3, although the comparison does not isolate their individual effects. We have updated the DDPO results and included the full DPOK comparison, mathematical formulations, and evaluation protocol in Appendix E.7 of the revised manuscript. ### Q5. Computational analysis We address this in the common computational-cost response. ### Q6. Normalization constant $Z(y)$ and practical approximations Thank you for raising this question. Eliminating $Z(y)$ from the score is **analytically exact**: it depends on the prompt but not the generated state, so $\nabla_z\log Z(y)=0$. Under the balance and compatible-boundary conditions of Theorem 1 (Appendix C.2), the continuation weights in Eq. 5 telescope to recover the normalized tilted distribution. Practical error arises when learned continuation scores, restricted LoRA transitions, and finite training approximate these conditions; it can affect both target suppression and benign proportions. Our **controlled 2D Gaussian mixture experiment** (shown in R4-Q2 and detailed in Appendix F, Table 6 and Figures 7 and 8) has a known analytical partition function $Z$ and tilted density. JS divergence approaches the matched exact-sampler floor, with target suppression and benign preservation. This supports target recovery in the representable, prior-compatible toy setting; it does not guarantee exact recovery under practical diffusion approximations.
Reviewer_bfqPOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents TILDE, a concept unlearning method that aims to remove unwanted concepts without unnecessarily changing the model's original generation distribution. The main idea is to define the desired distribution after unlearning through a KL-based formulation, which leads to an energy-tilted distribution that suppresses the target concept while staying close to the pretrained model. The authors use residual ∇\nabla-GFlowNet to approximate this distribution and show promising forgetting and preservation results across several diffusion and rectified-flow models.

Critical strengths

1. Clear motivation and principled formulation.

I find the motivation of this paper clear and convincing. Rather than focusing solely on suppressing unwanted concepts, the authors consider what the model should generate after unlearning. I particularly like the idea of explicitly defining the desired post-unlearning distribution through a KL-based formulation. The use of residual ∇\nabla-GFlowNet is also well motivated in this context, providing a natural way to approximate the proposed energy-tilted distribution

2. Comprehensive experiments and careful analysis.

The paper provides extensive experiments across different concepts, baselines, and generative architectures, showing promising forgetting–preservation trade-offs. I also appreciate that the authors evaluate both related and general retention, rather than focusing only on forgetting performance. The ablation studies, alternative concept scorers, and detailed theoretical discussions in the appendix further strengthen the paper

Critical weaknesses

Limited evaluation of practical scalability

While I find the proposed formulation and experimental results convincing, I have some concerns about the practical scalability of TILDE. The method requires a separate optimization process for each forget concept, with the reported SD v1.5 training taking 2–5 hours per concept on two H100 GPUs. In addition, the concept energy requires threshold and scale calibration when applied to new concepts or base models.

I appreciate that TILDE introduces no additional inference overhead, which is a practical advantage. However, the experiments focus on single-concept unlearning, leaving it unclear how well the method would scale when many concepts need to be removed or when unlearning requests arrive over time. In particular, I would be interested in understanding whether multiple concepts can be handled within a single optimization process and how the associated calibration and preservation challenges would change

Questions for authors

Q1. Preservation of non-target concepts with high CLIP similarity

The paper evaluates related and general retention, which I appreciate. However, I wonder how TILDE performs on non-target concepts that are semantically unrelated to the forget concept but have high CLIP similarity to it, as considered in the Pinpointness evaluation of HUB [1]. Since such concepts may also receive high forget energy, could the authors evaluate whether TILDE preserves them? I think this would provide additional insight into the selectivity of the proposed energy-based preservation mechanism.

[1] Holistic Unlearning Benchmark: A Multi-Faceted Evaluation for Text-to-Image Diffusion Model Unlearning

3 comments

W1. Threshold calibration and practical cost

Thank you for raising this point. The threshold and the energy scale play different roles in our method. The threshold determines whether an image contains enough evidence of the target concept to receive a forgetting penalty, whereas the scale determines the strength of that penalty. Since CLIP similarities can vary across concepts and base models, checking the threshold is an important part of applying TILDE to a new concept.

This calibration does not require another unlearning run. As explained in our response to R2-W1/W2, we use context-preserving target-absent prompt pairs inspired by Six-CD (CVPR 2025), generate reference images with the frozen base model, and calculate their CLIP similarities to select the appropriate threshold, taking only approximately 5 minutes. This is a one-time cost for each new concept or base model and requires no gradient updates or additional training.

Scale (λscale\lambda_{\mathrm{scale}}) and sharpness (α\alpha) do not require per-concept recalibration: they are fixed hyperparameters across concepts for a given base model/scorer configuration.

W2. Joint and continual unlearning

TL;DR: Without using any explicit retain datasets or retain replay, TILDE scales to both joint 50-celebrity erasure (87.0% unlearning, 90.0% general retention) and sequential 5-task continual unlearning (82.0% unlearning, 88.0% general retention) without catastrophic forgetting.

Thank you for this suggestion. TILDE’s energy-tilted formulation naturally extends to both joint multi-concept unlearning (via an additive joint energy E~joint(x)=∑c=1KE~c(x)\widetilde E_{\mathrm{joint}}(x)=\sum_{c=1}^{K}\widetilde E_c(x), corresponding to the product of multiplicative tilts ∏c=1Kexp⁡(−E~c(x))\prod_{c=1}^{K} \exp(-\widetilde E_c(x)) in a single optimization run) and continual unlearning. We conducted new experiments on both setups:

Joint unlearning. We unlearned 50 celebrity identities simultaneously in a single optimization process, evaluating all 50 target energies on every generated image with a single shared celebrity threshold τ\tau (per-concept thresholds τc\tau_c are equally supported) and zero explicit retain training examples. Using GCD on 500 images for erasure and Qwen2.5-VL across 60+ broad non-celebrity concepts for retention, TILDE achieves highly competitive performance against SPEED (ICLR 2026):

ModelUnlearning (%) ↑General retention (%) ↑
SPEED96.2%89.0%
TILDE (Ours)87.0%90.0%

Continual unlearning (sequential requests without retain replay). Starting each unlearning request sequentially from the previously merged checkpoint and optimizing the current target’s forget energy without any retain replay or continual regularization:

  • Mean Unlearning Accuracy (across 5 targets): 82.00%

  • Related Retention (RRaccRR_{\mathrm{acc}}): 78.49%

  • General Retention (GRaccGR_{\mathrm{acc}}): 88.00%

These results indicate that TILDE avoids catastrophic forgetting of benign capabilities in the evaluated continual setting, preserving related and general concept generation without retain replay. This addresses the cumulative retention collapse reported in the LACU study for several existing unlearning methods under sequential requests.

An important feature of both extensions is that they use no explicit retain-training examples or retain replay. SPEED uses retain-concept text embeddings to constrain its edits, while continual methods such as LACU (ECCV 2026) and Continual Unlearning (ICLR 2026) rely heavily on explicit retain sets and weight-drift regularizers to prevent degradation. These preservation mechanisms are natural additions to TILDE, as shown in the main paper Table 4. Both joint and continual unlearning experiments are detailed in Appendix J (Tables 9 and 10) of the revised manuscript.

Q1. Preservation of similar non-target concepts

Thank you for suggesting the HUB Pinpoint-ness evaluation. As detailed in our response to R2-W3 and Appendix H (Table 7), we evaluated TILDE on HUB’s released high-CLIP-similarity non-target concepts, which shows better preservation of neighboring concepts and surrounding prompt content compared to baselines while effectively suppressing the target concept.

MethodTarget proportion ↓Target aesthetics ↑General alignment ↑Selective alignment ↑Pinpoint-ness ↑Multilingual ↓Attack ↓
Base (SD1.5)0.655.3430.1720.560.590.490.49
SLD0.235.2960.0770.570.500.100.19
AC0.305.3080.1270.600.520.190.28
ESD0.145.265-0.0820.550.260.070.15
UCE0.255.3390.1930.580.530.110.25
SA0.175.102-0.3220.500.130.070.16
RECELER0.085.2700.0060.490.310.030.10
MACE0.145.262-0.3450.560.300.110.12
TILDE (Ours)0.065.401-0.0090.630.580.030.04

Thank you for the clarifications and additional experimental results. I think these results resolve my concerns and strengthen the paper. I also appreciate authors' effort and time! I will maintain my positive rating.

Reviewer_iBQrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper propses a novel concept unlearning for text-to-image generation model (Diffusion models). The authors criticizes prior works' heuristic algorithmid design as they lack foundational intuition and leads to over or insufficient unlearning. the paper argues that existing works does not preserve distributional fidelity. The distribution of remaining concepts shoud be identical to the pre-trained or retrained reference model, but the existing works does not address this and their algorithm eventually skews the distribution. To address this, the paper positions the unlearning problem into distributional alignment problem. The paper uses a pre-trained CLIP to capture the desired distributional shift in its gradient, and uses ∇\nabla-GFlowNet for optimization backbone. Experimental result shows that the proposed method achieves good trade-off of unlearn efficacy and model utility, measured by an LLM Judge.

Critical strengths
  1. Problem setting and formulation is well argued and has a strong mathematical foundations. The paper proposes a solid idea on steering the model to the distribution alignment, and connects it to defining a closed-form optimization target (Equation 2).

  2. The paper offers a set of rigorous experiments, including retention evaluation over 60+ concepts.

  3. The paper demonstrates the scalability over the other diffusion models.

Critical weaknesses
  1. Each category receives a \tau and experiments used identical tau for different concepts (apple, banana, golf ball, cat, dog). However it is not clear if CLIP's similarity scale are identical among them. For example, CLIP may assign higher score to images with cats than golf ball, in both presence and absence of the concepts. If so, τ\tau can cause over-unlearning for some concepts.

  2. It is unclear whether a practitoner unlearning a new concept can re-use the same τ\tau or needs a new calibration. While the authors have discussed this in the limitations, this is an essential part of their proposed method and hence needs a clarification.

  3. Related concepts were chosen by GPT-4, and it may have less conceptual similarity in the model. For example, Squirtle, Charmander or Jigglypuff may share less visual similarity with "Pikachu". They may less likely to be flagged by τ\tau. Thus the authors should offer CLIP-score proximity of the related concepts to the target concepts.

  4. The paper seems not comparing the efficiency of the proposed method over the baselines. The authors can offer running time of the algorithm and the paper needs to compare them with baselines.

  5. The paper seems not have any robustness analysis such as relearning attack.

Questions for authors
  1. Both RR and GR remains steady on Figure 3. Does that mean that related or general concpets are not affected by τ\tau?
6 comments

W1. Different CLIP similarity scales across concepts

Thank you for raising this important point. We fully agree that CLIP cosine similarities are not calibrated probabilities and can exhibit concept-specific and model-specific biases. Our calibration procedure is inspired by the dual-version evaluation idea in Six-CD as detailed in Appendix E.6. We construct a second version of each prompt by removing the target concept while preserving the surrounding context. For example, “Pikachu standing beside a lake at sunset” can become “a lake at sunset.” We then generate reference images from these target-absent prompts using the frozen base model and measure the remaining CLIP similarity. This provides a reference for the similarity that can arise from the context even without the target. Empirically on SD v1.5, concepts within the same semantic granularity/category showed similar thresholds. When moving to different base models (SD v1.4, v2, SD3, FLUX.1-dev) or scorers, this calibration adjusts τ\tau to the respective model’s score distribution.

W2. Calibration for a new concept or backbone

For a new concept or a different base model, the practitioner should validate or recalibrate the threshold. The same numerical value need not transfer, because both the concept and the generator can change the distribution of similarity scores. The existing category threshold can be used as a starting point.

The calibration step requires generating a set of reference images and evaluating CLIP similarities; it does not require training an additional detector or updating the generative model. Generating context-preserving target-absent prompts from forget prompts (following Six-CD) is automated using an LLM. We generate a small batch of reference images (16 images per prompt) using the frozen base model and compute their average CLIP score to set the baseline threshold τ\tau.

Because this process only involves generating a handful of images and computing forward-pass CLIP cosine similarities on a few prompts with no gradient updates or detector training, it runs automatically in just a few minutes(approx 5 min) and adds negligible overhead.

W3. Related concepts and CLIP proximity

Our choice of related concepts follows the motivation in EraseBench (ICCV 2025): unlearning can affect neighboring concepts, so retention should include concepts likely to exhibit this ripple effect. We use GPT-4o-mini to propose these semantic neighbors. The reviewer raises a valid point that semantic neighbors may not always have the highest CLIP similarity to the target. To directly evaluate preservation on non-target concepts that have high CLIP similarity to the forget concept, we ran the HUB Pinpoint-ness and Selective Alignment evaluations, as well as general alignment evaluations (see the comparison table in R3-Q1). The results show that TILDE better preserves high-CLIP-similarity neighboring concepts and the surrounding content of a target prompt compared to baselines, while effectively suppressing the target concept.

W4. Runtime comparison

We address this in the common computational-cost response.

W5. Robustness to relearning

Thank you for raising this very important point. We agree that a useful unlearning method should remain effective when the model is subsequently fine-tuned. To study this, we adapted the target-absent curriculum from The Illusion of Unlearning (CVPR 2025) to our SD1.5 checkpoints, evaluating four concepts shared between our benchmark and that study. The model is fine-tuned through ten curriculum stages, and we evaluate the same 100 held-out prompt/seed pairs throughout the curriculum using Qwen2.5-VL-7B.

We conducted the relearning experiment on four concepts evaluated in both our work and The Illusion of Unlearning. After completing all ten fine-tuning stages, TILDE achieves 91% average unlearning accuracy across these four concepts.

One major reason for this persistence is that TILDE performs anchor-free distributional erasure rather than steering trajectories toward a fixed anchor token, making it more resilient to benign fine-tuning. Recent works like AGE and LACU have similarly argued against fixed anchors in favor of adaptive anchor concepts. Further studies are needed to explore the exact mechanics of anchors under revival, which is beyond the scope of this work. In the final manuscript, we have included this relearning evaluation in Appendix I (Table 8).

Q1. Interpreting the retention curves in Figure 3

The related and general retention scores do change with the threshold, though the broad yy-axis scale makes the variations appear compressed. Considering the two sides of the selected threshold (τ=0.22\tau=0.22 for Pikachu) illustrates the trade-off:

  • Threshold too low (τ<0.22\tau < 0.22 or unthresholded): Weak target evidence in benign images triggers the penalty and degrades retention. Raising τ\tau from 0.190.19 to 0.220.22 improves related retention from 7272\\% \to 76\\% and general retention from 8585\\% \to 88\\% while maintaining 9999\\% unlearning accuracy; removing the threshold entirely collapses related and general retention to 88\\% and 4949\\%.

  • Threshold too high (τ>0.22\tau > 0.22): Target images remain unpenalized; at τ=0.25\tau=0.25, retention rises slightly (8383\\% related, 8888\\% general) but unlearning accuracy drops sharply to 2323\\%.

Thus, τ\tau governs the Pareto trade-off between forgetting and retention, with τ=0.22\tau=0.22 capturing the optimal knee. We have added the exact numerical table (Table 5 in Appendix E.6) and revised Figure 3 to make this clearer.

We would like to further highlight the persistence of TILDE’s unlearning under subsequent fine-tuning. After completing all ten stages of the target-absent curriculum, TILDE maintains an average unlearning accuracy of 91%, substantially higher than the baseline accuracies reported in The Illusion of Unlearning, as summarized below.

MetricESD-uESD-xUCECASalUnMACESPMRECELEREDiffSATILDE
Post-fine-tuning UA ↑64.00%46.00%67.56%30.50%40.75%62.71%29.80%75.00%44.33%37.33%91%
Reviewer_eeVsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The authors of TILDE stress that in current unlearning works, to push for higher benchmark scores, evaluations mistake failure modes like mode collapse, visual distortion or over-erasure of neigbouring concepts as successful forgetting; in the process, the model's ability to generate other benign concepts also degrades. Apart from the traditional forgetting and retain set preservation, the authors emphasize that post-unlearning distribution should be close to the pretrained or retain-only distribution at per-prompt, outside of the forget set (their one constraint). They choose the conditional distribution which is closest to the pretrained model as per KL divergence metric. The authors perform this by tilting each prompt-conditioned pretrained distribution according to the forget energy. However, the constraint here is defined on the final generated image, and a diffusion model generates images through several denoising steps. So, the authors devise a method to transfer the corrections at every denoising step, such that model learns to steer clear of the concept. Primarily, they are learning how the original denoising changes to check desired distribution. They state that the residual continuation weight lowers itself when future denoising trajectories is likely to lead to an unwanted image, starting from a noisy latent. The corrections are measured by residual GFlowNet, by comparing transition scores between pretrained and LoRA adapted models. Each timestep they estimate a clean image, measure concept-related forget energy and form a gradient which encourages the model to move away from the unrequired concept. Experiments on objects, characters, artistic styles and nudity show that TILDE achieves strong forget accuracies and maintains good retention of related and unrelated concepts.

Critical strengths
  1. The paper highlights a major problem in unlearning literature: related or unrelated concepts should not be harmed while removing an unwanted concept, and the post-unlearning distribution should be close to the pretrained one's outside of forget set. The authors develop a mathematical framework based on this insight, choosing the distribution which suppresses the forget concept while being the closest to pretrained model distribution.

  2. TILDE shows the strongest forgetting and retain performance across several baselines.

  3. The authors also demonstrate how closely the distribution of unlearnt model matches the retain-only reference, using FADE.

  4. Ablations on objective components, residual network, forgetting threshold etc contributed well to the method's claims.

Critical weaknesses
  1. It would be helpful to provide a background for GFlowNet before delving into the method in Section 3. Terms are not properly explained before being used in the method. Also, many of the terms and definitions are difficult to follow throughout the paper.

  2. TILDE is expensive (3-5 hours per concept). Compute cost comparison wrt other methods are not provided.

  3. Ablations are primarily conducted on one concept (Pikachu). It would be helpful if ablations were shown on multiple concepts.

  4. Additional tests wit adversarial prompts, implicit prompts, indirect descriptions would be useful.

Questions for authors
  1. What happens in the case of adversarial prompts?

  2. How much improvement does GFlowNet provide wrt CLIP-based forgetting energy and threshold?

  3. Is there a trade-off available between the training time and forget/retain performance?

5 comments

W1. Background and terminology for GFlowNets

Thank you for pointing this out. Due to strict main-paper page limits, we focused the main text heavily on the unlearning formulation and empirical evaluation, placing the GFlowNet preliminaries in Appendix B.2 and C.1. In the revised manuscript, we have added more background on GFlowNets.

W2. Computational efficiency

We address this in the common computational-cost response.

W3. Breadth of the ablations

Thank you for this suggestion.

  • Table 3 (Objective Ablation): Table 3 already shows a clear difference between the objective variants, and we agree that including more concepts strengthens this conclusion. We extended this objective ablation to 4 additional concepts (Appendix E.7), where TILDE with residual ∇\nabla-GFlowNet consistently achieves the best trade-off between forgetting and retention with similar trends. In the final manuscript, we will include the results for these additional concepts.

  • Table 4 (Effect of Retain Prompts): We have now evaluated this ablation across 6 concepts. The table shows higher related retention for TILDE+RR and higher general retention for TILDE+GR than the TILDE. We will update the manuscript with results for the remaining concepts once available.

    MethodUnlearning ↑Related retention ↑General retention ↑
    TILDE0.960.710.86
    TILDE + RR0.970.740.87
    TILDE + GR0.970.680.88
  • Multi-Concept Ablations Already in the Appendix: Our supplementary material already reports comprehensive ablations across all 12 benchmark concepts (Objects, Characters, and Styles) for alternative concept scorers (Appendix L.5, Table 13) and across five generative backbones (SD v1.4, SD v1.5, SD v2, SD3, and FLUX.1-dev in Appendix M, Table 14).

W4 and Q1. Adversarial, indirect and multilingual prompts

TL;DR: On HUB (ICCV 2025) and I2P, TILDE achieves the lowest attack detection (0.04 on Ring-A-Bell) and multilingual detection (0.03 across 5 languages) while preserving benign content (0.58 pinpoint-ness vs. 0.59 base).

Thank you for raising this important point. We agree that an unlearned model should suppress the target even when the prompt differs from the ordinary prompts used during training. In Appendix K (Table 11), we already evaluated nudity unlearning on I2P, a benchmark specifically composed of implicit and adversarial real-world prompts that evoke NSFW content without explicit keywords. Using a lightweight classifier-based forget energy, TILDE outperforms non-adversarial baselines while remaining competitive with dedicated adversarial-training methods like R.A.C.E.

Following the Holistic Unlearning Benchmark (HUB) (ICCV 2025) protocol, we also evaluated TILDE across all shared concepts between our benchmark and HUB using 1,000 images per prompt panel on (i) Ring-A-Bell adversarial transfer attacks (ICLR 2024) and (ii) five translated languages (Spanish, French, German, Italian, Portuguese; testing implicit cross-lingual generalization when trained only on English descriptors). Across the shared concepts (see the full HUB table in R3-Q1), TILDE reduces the target generation proportion (↓\downarrow) to 0.06 (vs. 0.65 base), Ring-A-Bell attack detection (↓\downarrow) to 0.04 (vs. 0.49 base), and multilingual detection (↓\downarrow) to 0.03 (vs. 0.49 base).

A further point we would like to emphasize, as highlighted in the HUB study, is that adversarial robustness and benign retention form a strict trade-off: methods that achieve low attack detection via aggressive parameter suppression typically suffer severe collateral damage on retain concepts. TILDE performs well on both, achieving strong target suppression while preserving benign content, as shown in our HUB pinpoint-ness, selective alignment, and general alignment results in R3-Q1.

Q2. How much improvement does GFlowNet provide with respect to the CLIP-based forgetting energy and threshold?

Thank you for this question. The CLIP-based forgetting energy and threshold specify which generated outputs should be suppressed and how strongly they should be penalized. The GFlowNet formulation determines how this signal is used to update the denoising process so that the final samples approximate the tilted distribution. Its benefit is therefore not only to forget the target, but also to preserve the model’s other capabilities while doing so.

Table 3 examines this distinction while keeping the thresholded CLIP energy and LoRA parameterization fixed. Compared with the existing reward-weighted score-matching surrogate, TILDE achieves overall best numbers. These results show that the forgetting energy and threshold alone do not explain the overall forgetting-retention performance. Also the component ablations further clarify this contribution.

Q3. Trade-off between training time, forgetting and retention

Yes. During single-concept unlearning, as training progresses, unlearning accuracy UaccU_{\mathrm{acc}} improves rapidly as the forget energy E~C(x)\widetilde{E}_{\mathcal{C}}(x) decreases, while general retention GRaccGR_{\mathrm{acc}} drops only marginally because images with sC(x)≤τs_{\mathcal{C}}(x) \le \tau receive zero direct terminal-energy gradient. In practice, we monitor the online forget energy E~C(x)\widetilde{E}_{\mathcal{C}}(x) during training and stop when the forget energy converges near zero and sampled training trajectories no longer generate the target concept.

The timing comparison for all the baselines are done on different GPUs, hence it is difficult to compare. Can the methods or part of the methods be transported to the same one for better comparison?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Common Concerns

We thank the reviewers for carefully reading our paper and providing constructive feedback. We are glad that all reviewers appreciated the importance of considering both what an unlearned model should forget and what it should preserve, found our mathematical formulation well-motivated, and recognized the breadth of our experiments across base models and scorers. Your suggestions genuinely helped us strengthen the manuscript by demonstrating TILDE’s strong resilience to relearning, adversarial and multilingual robustness, and effectiveness in practical deployment settings such as continual and joint multi-concept unlearning requests. Since computational cost was raised by all, we first address it in a common response below.

Common response: Computational cost

Addresses R1-W2, R2-W4, R3-W1, and R4-Q5.

TL;DR: TILDE requires a one-time offline provider update of $\sim$4 H100 GPU-hours per concept (comparable to or lower than recent distillation- and robustness-based unlearning methods), with zero inference-time latency or memory overhead at deployment. For the practical use case of unlearning algorithms, this computational cost is not a bottleneck and is well worth the trade-off given the strong forgetting, retention, and robustness benefits TILDE provides.

Thank you for raising this point. Our main focus in TILDE is to achieve effective unlearning while preserving the useful capabilities of the original model. We do agree that computational cost is an important consideration. But one major thing we want to emphasize is that unlearning algorithms are primarily executed by model providers and institutions that pretrain foundation models and must subsequently erase specific concepts for copyright, privacy (e.g., GDPR), or safety compliance. For these institutions, a one-time cost of ~4 GPU-hours is minor and because trajectory rollouts in Phase 1 of Algorithm 1 are independent across prompts, data parallelization across 8 GPUs can further reduce wall-clock time toward ~30-45 minutes. What institutions cannot afford is incomplete erasure or silent degradation of general model capabilities, where TILDE provides a superior Pareto trade-off. Also, here the computational cost is a one-time expense, and the resulting model discards both the concept scorer and the auxiliary continuation network. Inference uses the standard base sampling schedule with zero extra denoising steps, zero runtime guardrails, and zero additional latency or memory. Because deployed text-to-image models serve millions of queries, avoiding per-query inference overhead is paramount for practical scaling.

We do agree that the computational cost of our method is significantly higher than editing based methods but the benefits of our approach outweigh the costs as seen with the superior performance. There are also other methods like distillation based ones that have comparable or higher computational costs than TILDE. A typical CLIP-based TILDE unlearning run takes approximately two hours on two H100 GPUs, or four H100 GPU-hours. With the Qwen VLM as the concept scorer, the cost increases, and for broad categories like Cat and Dog, convergence is slightly slower.

To put the training cost in context, we have collected the following reported runtimes for single concept unlearning. We retain the hardware and task descriptions because these measurements come from different experimental settings.

MethodReported computation
TILDE4 H100 GPU-hours
LACU (ECCV 2026)4.5 H100 GPU-hours
Selective Amnesia (NeurIPS 2023)20 h on 4 RTX A6000s
MACE (CVPR 2024)2 A6000 GPU-hours
Meta-Unlearning (ICCV 2025)24 A6000 GPU-hours
EraseFlow(+AdvUnlearn) (NeurIPS 2025)0.05 (24.3 w/AdvUnlearn) H100 GPU-hours
SFD (ICLR 2025)>= 24 H100 GPU-hours

These results show that the training cost varies substantially across methods. TILDE requires more optimization than fast editing methods, while other approaches also devote substantial computation to preserving the model during unlearning. Our motivation is that an offline update with a useful forgetting-preservation balance can be valuable for the model-provider setting described above, especially when it leaves deployment cost unchanged.

We agree that reducing the update time would be even more beneficial for practicality, particularly when requests are frequent. In the final manuscript, we will report more detailed training cost and runtime comparisons with other methods, and discuss potential avenues for further optimization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.