- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
- This paper investigates the idea of machine unlearning in terms of tilted distributional erasure. Based on the idea of ∇-GFlowNet, this paper formalizes unlearning in terms of KL minimization and a penalty function for unwanted data generation. The paper includes various experimental scenarios of unlearning.
Critical strengths
- The paper investigates an important question in the unlearning domain.
- The paper presents well-supported training strategies by defining unlearning in terms of KL divergence.
- The paper includes various experiments to back up the proposed method.
Critical weaknesses
Please refer to the Questions below.
Questions for authors
-
Using residual score or one-step reward gradient is already proposed in ∇-GFlowNet as the author mentioned. Thus, I believe that the novelty of this paper is marginal. What is the main difference compared to ∇-GFlowNet in the methodology?
-
The right semantic example of Figure 1 shows the optimal results of the proposed method against other methods. Can we observe the similar outputs in toy or real data?
-
Can the authors explain in detail why preserving the ratio of benign data is optimal for machine unlearning? If the training data is changed due to the deletion of forget data, it is reasonable that the final output distribution is not akin to the total distribution of the original model with full data even if we delete the forget region in the distributional sense. If we consider the understanding of optimal transport, it might be optimal to transport all the mass of forget data into the near mode.
-
If we choose KL minimization as a criterion, I agree that an energy-tilted distribution can be utilized. However, there are many methods that implement KL minimization, e.g., KL-regularized reward optimization. The authors compare against DDPO in Table 3, but it does not clarify whether DDPO uses the same KL-regularized objective, making it difficult to determine whether the improvement comes from the objective choice or the optimization.
-
The computational analysis against the existing method is not well provided, which is one of the main problem of energy-based models.
-
Dealing the normalization constant in the energy-based model is very important. The authors avoid explicitly computing the normalization constant by matching gradients. However, how do the approximations in this procedure affect the resulting distribution, especially in unlearning?
2 comments
Q4. KL-regularized reward optimization and the role of GFlowNet
TL;DR: While output-level KL-regularized RL shares the same theoretical Gibbs optimum, policy-gradient methods (DDPO/DPOK) rely on scalar trajectory returns and struggle with the forgetting-retention trade-off, whereas TILDE leverages first-order differentiable energy gradients and step-wise residual balance.
Thank you for pointing this out. We agree that it is important to separate the distributional objective from the optimization method used to learn it. With the tilt strength absorbed into , as in the paper, KL-regularized reward maximization
-D_{\mathrm{KL}}\\!\left(p(\cdot\mid y)\\,\\\|\\,p_{\mathrm{pre}}(\cdot\mid y)\right)$$ has the same Gibbs optimum $p^*(x\mid y)\propto p_{\mathrm{pre}}(x\mid y)e^{-\widetilde E_{\mathcal C}(x)}$ as TILDE’s practical target (Eq. 4). This equivalence is at the output-distribution level. DPOK uses trajectory KL, which upper-bounds output KL and also constrains denoising paths; with a fixed prior and restricted transitions, its optimum need not have the same output marginal. Using the same TILDE setup, we evaluated both: - **DDPO:** Maximizes the terminal forgetting reward via clipped importance-sampling policy-gradient updates over denoising steps, without an explicit KL penalty to the frozen pretrained model. - **DPOK:** Uses the same terminal forgetting reward with an explicit trajectory-level KL penalty against the frozen pretrained model. As shown in Table 3, both DDPO and DPOK achieve strong forgetting, with unlearning accuracies of **0.97** and **0.99**, respectively. However, their related/general retention scores are **0.45/0.82** for DDPO and **0.59/0.79** for DPOK. TILDE achieves **0.99** unlearning accuracy while maintaining higher related and general retention of **0.76** and **0.88**, respectively. These results show that TILDE provides a better balance between forgetting and preservation in this comparison. **Residual $\nabla$-GFlowNet** offers two relevant mechanisms that may contribute to this performance: 1. **First-Order Differentiable Energy Gradient.** The reward terms in DDPO and DPOK use score-function policy gradients weighted by scalar returns, without differentiating the forget energy with respect to the generated image. In contrast, TILDE uses the energy gradient through the predicted clean image, $-\bar\alpha_t\nabla_{z_t}\widetilde E_{\mathcal C}(\hat x_\theta)$, together with the learned continuation residual $g_\phi(z_t,t)$ (Eq. 8), supplying an explicit local direction for reducing concept evidence during denoising. 2. **Step-Wise Residual Detailed Balance.** The forward and reverse losses encourage step-wise consistency between the local transition-score correction and the continuation score relative to $p_{\mathrm{pre}}$ (Eqs. 6 to 9). Below the threshold $\tau$, the direct energy-gradient term vanishes identically, avoiding a direct forgetting force on low-evidence estimates. The continuation term $g_\phi$ and shared parameter updates can still change these transitions, so this property alone does not guarantee benign preservation. These design differences may contribute to the balance between forgetting and preservation observed in Table 3, although the comparison does not isolate their individual effects. We have updated the DDPO results and included the full DPOK comparison, mathematical formulations, and evaluation protocol in Appendix E.7 of the revised manuscript. ### Q5. Computational analysis We address this in the common computational-cost response. ### Q6. Normalization constant $Z(y)$ and practical approximations Thank you for raising this question. Eliminating $Z(y)$ from the score is **analytically exact**: it depends on the prompt but not the generated state, so $\nabla_z\log Z(y)=0$. Under the balance and compatible-boundary conditions of Theorem 1 (Appendix C.2), the continuation weights in Eq. 5 telescope to recover the normalized tilted distribution. Practical error arises when learned continuation scores, restricted LoRA transitions, and finite training approximate these conditions; it can affect both target suppression and benign proportions. Our **controlled 2D Gaussian mixture experiment** (shown in R4-Q2 and detailed in Appendix F, Table 6 and Figures 7 and 8) has a known analytical partition function $Z$ and tilted density. JS divergence approaches the matched exact-sampler floor, with target suppression and benign preservation. This supports target recovery in the representable, prior-compatible toy setting; it does not guarantee exact recovery under practical diffusion approximations.
Q1. Methodological difference
We adopt the residual score-matching idea from ∇-GFlowNet because it provides a formulation for approximating energy-tilted distributions in latent space; we do not claim ∇-GFlowNet itself as our contribution. Rather, our core contributions lie in formulating concept unlearning as an anchor-free distributional alignment problem and adapting ∇-GFlowNet with critical unlearning-specific mechanisms:
Posing concept unlearning as constrained distributional alignment (Proposition 1): Whereas ∇-GFlowNet was designed for proportional sampling from unnormalized reward densities, concept unlearning requires suppressing a specific target while preserving benign generation. We formalize this via constrained KL minimization (Eq. 1), deriving the prompt-conditioned Gibbs target.
Adapting ∇-GFlowNet with a thresholded forget energy & multi-architecture/multi-scorer extensions: Directly applying ∇-GFlowNet with a continuous penalty destroys benign retention (Figure 3). We introduce a thresholded forget energy with a zero-penalty region and extend the stochastic transitions to Rectified-Flow transformers (SD3, FLUX) and VLM/classifier scorers.
Proposing Anchor-Free Unlearning: Several prior methods map forget concepts to fixed anchors, while recent literature (AGE (ICLR 2025), LACU) motivates adaptive anchors to improve preservation. TILDE instead defines a target without prescribing a replacement anchor. Our experiments show strong retention and robustness to relearning, although they do not isolate anchor removal as the cause of these gains.
Q2. Controlled evidence for the distributional illustration
Thank you for raising this point. Yes. We evaluated TILDE on a controlled 2D mixture of eight overlapping Gaussian components (using exact mixture reverse transitions and a compatible Gaussian prior) where one target component is suppressed by ρ=0.5 (moderate) or ρ=0.1 (strong), trained for 30,000 steps across 5 fresh seeds per strength and evaluated on 1,000,000 samples per seed. Across both strengths, TILDE suppresses the target mode while preserving the relative proportions of all seven benign modes, reducing the 2D Jensen-Shannon (JS) divergence from 0.0053→0.00109 for ρ=0.5 (exact-sampler floor: 0.00107) and from 0.0230→0.00110 for ρ=0.1 (exact-sampler floor: 0.00107). We have included the full experimental setup, quantitative table, 2D endpoint density heatmaps, and unwrapped angular marginal curves in Appendix F (Table 6 and Figures 7 and 8) of the revised manuscript.
Q3. Distributional preservation, retraining, and the connection to FADE
TL;DR: Without access to an infeasible retain-only retrained model, the Gibbs tilted distribution pC∗(x∣y) is the minimum-KL deviation from ppre(x∣y) that proportionally reweights zero-energy benign outputs without altering their relative probabilities; empirically, TILDE also achieves the best FADE distributional alignment scores (Table 1).
Ideally, an unlearned model should match a retain-only model pretain(x∣y) retrained from scratch, which is computationally infeasible when it comes to large models. As discussed in FADE (Cho et al., 2025), evaluating distributional alignment (DKL(pretainy∥punlearny)+DKL(punlearny∥pretainy)) is essential because point-wise erasure metrics hide distributional drift. Without access to pretain, under the stated regularity conditions, the minimum-deviation distribution satisfying the forget constraint relative to ppre(x∣y) is the tilted distribution pC∗(x∣y)=ppre(x∣y)exp(−βEC(x))/Zβ,C(y). Suppressing the target region (E~C(x)>0) and renormalizing by 1/Zβ,C(y) multiplies the densities of zero-energy benign outputs (E~C(x)=0) by the same factor, preserving their pretrained conditional relative probabilities. This is proportional reweighting, not a nearest-mode transport rule; context preservation depends on what ppre(x∣y) supports. Fixed-anchor methods instead prescribe a replacement concept, which can introduce collateral changes; anchor choice alone does not establish whether revival occurs. We also empirically evaluate distributional agreement using FADE rather than relying only on the analytical argument. The results show that TILDE achieves the best scores in our table while achieving the strong unlearning performance.