acceptodds
Under review as a conference paper at ICLR 2027

Continual Data Unlearning in Diffusion Models via Transition-Based Regularization

Abstract

Data unlearning in diffusion models aims to remove the influence of specific training examples without suppressing the broader concepts they represent. However, when deletion requests arrive sequentially, updates for new requests can degrade generative utility and undermine earlier deletions. We propose a continual data unlearning framework that uses completed deletion transitions as directional references to regularize future updates. For each request, we record changes in denoiser responses on the same fixed noisy inputs before and after unlearning. Rather than matching full post-deletion responses, we apply a one-sided penalty that discourages reversal along the recorded directions relative to the post-deletion references, while leaving orthogonal response changes and progress beyond these references unpenalized. To keep storage independent of the number of requests, we maintain a fixed-capacity bank of representative transition records. Records are selected based on the local sensitivity of progress along their recorded directions to parameter updates, allowing them to be retained even when their penalties are inactive. Empirical evaluations show that the proposed framework achieves a better balance between deletion persistence and generative utility than existing unlearning baselines as requests accumulate, using only a small transition memory.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine Unlearning, Continual Data Unlearning, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_DC88OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a continual data unlearning method in diffusion models. The proposed method first records denoiser response changes on the same fixed noisy inputs before and after unlearning; then in subsequent updates, only the direction aligning with the recorded directions is penalised. Experiments show the effectiveness of the proposed method.

Critical strengths
  • The motivation is clear. In continual data unlearning, subsequent updates may degrade image quality or weaken earlier deletions; this paper proposes a method based on common strategies in continual learning to perform effective continual data unlearning.
  • The experimental results on CIFAR-10 and CelebA-HQ demonstrate the effectiveness of the proposed method.
Critical weaknesses

Since data unlearning is for deletion rights, would it be reasonable if the proposed method needs to permanently store noisy copies of deleted samples as probes?

Questions for authors
  • It would be better to include an analysis on whether the adversary could reconstruct samples from the probes.
  • In Table 3, the reported figures are averages after 20, 50, and 100 deletions; will earlier deletions be recovered after being abandoned (especially for 100 deletions), since the fixed-capacity memory protects the most recent deletions?
  • Minor: in line 178, the symbol Φ\Phi needs to be explicitly explained when it first appears, not in the appendix.
Reviewer_UBa2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors study continual data unlearning in Diffusion models. The authors argue that existing methods in continual setup face 2 major problems: degradation and reappearance. So here the authors propose a continual unlearning framework that uses completed deletion transitions as directional references and applies a one-sided penalty that discourages reversal along the recorded directions relative to the post-deletion references, while leaving orthogonal response changes and further movement along the recorded directions unpenalized. The framework is implemented with ReTrack and evaluated on CIFAR-10, CelebA-HQ, and Stable Diffusion.

Critical strengths
  • This is a promising direction, as in practical real-world deployment, we need scalable and reliable unlearning methods that can handle sequential requests.
  • Shows good performance in continual setup in CIFAR-10, CelebA-HQ, and Stable Diffusion.
  • The abalations provided are useful.
Critical weaknesses
  • The main contribution here is the continual unlearning framework and memory mechanism built around an existing unlearning objective, rather than proposing a new unlearning algorithm. Output preservation, asymmetric regularization, and gradient-based memory selection have relevant precedents in OGD, asymmetric functional regularization, and GCR. These connections should be cited, with a clearer explanation of what is distinct about using deletion-induced response transitions.
  • All experiments with the proposed framework use ReTrack, while SISS and EraseDiff are evaluated only as standalone baselines. Since the recorded transitions depend on the underlying unlearning algorithm, it remains unclear how much the benefit depends on ReTrack’s particular updates. An integration with another method, such as SISS, or a discussion of compatibility requirements and limitations would clarify the framework’s scope.
  • A simpler continual regularization baseline would strengthen the evaluation. Prior concept-level studies, including Lee et al. and George et al., demonstrate the value of regularization and retention mechanisms during sequential unlearning. So, a comparison with ReTrack using a simple parameter-drift penalty would help establish the additional benefit of the proposed transition-based mechanism.
  • Performance on longer, more diverse deletion streams remains unclear. Here the paper evaluates several datasets, it does not systematically test mixed streams involving artworks, celebrity photographs, and sensitive documents which is more important and practical too. In concept level unlearning literature too, we can see that methods performing well on celebrities could struggle with artistic styles, highlighting the importance of evaluating different request types. In practical deployment, a model may receive these different requests sequentially, raising questions about whether the fixed-capacity memory continues to protect earlier deletions. Testing mixed streams across memory capacities, with per-target recurrence and retained utility, would help address this concern.
  • Experiments limited to SDv1.4 only.
  • Memory selection does not consider how close a record is to reversal. The selection rule compares normalized gradient directions, but records with similar directions can have different protection needs because their current margins differ. Tables 3 and 5 provide useful capacity and selection ablations, but they do not show whether retained records adequately protect discarded targets.
Questions for authors
  • See above weaknesses.
  • Can the mechanism be applied to SISS without substantial changes?
  • To measure the target reappearance some simple metrics could be introduced to measure that?
  • FID numbers could be shown for SD experiments also.
Reviewer_Aj1fOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

the paper presents a novel continual unlearning method for diffusion models. The paper names every concept removal task a deletion request.

Before any deletion request the method defines the retain set and later removes each new target from it. For each request, the method runs ReTrack unlearning and records how the denoiser's outputs change on a few fixed noisy versions of the target (probes).

During later requests, a penalty applies only when these outputs move back toward their pre-deletion values. It leaves orthogonal changes and further progress unpenalized. A fixed size memory keeps the most representative probes, so memory does not grow with the number of requests.

Experiments were done with both unconditional using CIFAR-10 and CelebA-HQ datasets (images), and conditional diffusion models using prompt–image pairs from SISS.

Critical strengths

None

Critical weaknesses
  1. Evaluation has major gaps: the variance for the majority of the metrics is so high that the gains from ReTrack are within it. Only Figure 2 shows clear gains; however, no confidence intervals are shown and based on the tables they you overlap.
Questions for authors
  1. Why does CelebA-HQ SSCD from the table 1 differ so much from the CelebA-HQ SSCD for 50 deletions column from table 3?
  2. Why is the FID variance in some places 30%? How much samples were used for the FID computation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.