- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies data unlearning in flow matching generative models and proposes two methods, LFD and EBR. LFD uses loss decomposition and importance sampling to estimate the retain-only training objective when both retain and forget data are available. EBR instead uses proxy classifier scores to reweight the training distribution when direct access to forget samples is unavailable. The paper provides theoretical analyses of both methods and evaluates them on CelebA identity removal and MNIST digit-class unlearning.
Critical strengths
- The paper addresses a relevant problem by studying data unlearning in flow matching models and considering different levels of access to forget data.
- The paper provides mathematical derivations and includes exact retraining as an experimental reference. The results on CelebA and MNIST show that the proposed methods can reduce the generation of targeted content while preserving some retained generative capability.
Critical weaknesses
- The methodological novelty is limited, with substantial overlap with existing work. The loss decomposition and importance sampling mechanism in LFD closely resemble SISS (ICLR 2025), and the paper itself acknowledges their mathematical connection in Appendix E. The energy-reweighted objective of EBR directly corresponds to existing Energy-Weighted Flow Matching formulations (ICLR 2025) and is also closely related to the proxy-based unlearning approach in ContinualFlow (2025). Although these techniques are adapted to the proposed setting, the paper does not sufficiently establish its distinct technical contributions, particularly compared with EFM and related methods.
- The experimental evaluation is insufficient to support the main claims. LFD and EBR are evaluated separately on CelebA and MNIST, without cross-dataset validation or experiments on more challenging generative models. The CelebA experiments lack qualitative generation comparisons, and all reported results use Weighted LFD with an additional forget-ascent term rather than the theoretically unbiased Exact LFD. This makes it difficult to isolate the contribution of the proposed loss decomposition. On MNIST, the results remain substantially worse than exact retraining. For example, Table 2 reports a forgotten-class fraction of 10.7% for EBR compared with 1.0% for retraining. Furthermore, the ContinualFlow comparison relies on a local reimplementation rather than the original image-domain system. The presentation is also unnecessarily complicated, making the methodology and actual contributions difficult to follow.
Alberti et al., Data Unlearning in Diffusion Models (SISS), ICLR 2025 Zhang et al., Energy-Weighted Flow Matching for Offline Reinforcement Learning, ICLR 2025 Simone et al., ContinualFlow: Learning and Unlearning with Neural Flow Matching, ICML 2025
Questions for authors
- Could the authors clarify the main technical differences between LFD and SISS, and between EBR and existing Energy-Weighted Flow Matching or ContinualFlow methods? In particular, what new theoretical or algorithmic contributions do EBR's Eq. (4) and Eq. (6) provide beyond existing energy-weighted objectives?
- Why are results for Exact LFD with (\rho=0) not reported on CelebA? Since all main experiments use an additional forget-ascent term, how can the improvements be attributed to the proposed unbiased loss decomposition rather than this extra update? Could the authors provide the corresponding ablation and qualitative face-generation results?
- Could the authors explain the substantial performance gap between EBR and exact retraining, and provide sensitivity analyses for proxy quality or weighting parameters? Also, why is the comparison limited to a local reimplementation of ContinualFlow instead of a more faithful comparison with its original image-domain method?
3 comments
Rebuttal Part-2
| CelebA Joint-Six (, 256 states/step, 5 seeds) | Target Cosine @ Step 12 | Gap to Gold @ Step 12 | Gap to Gold @ Step 60 | Retained FID @ Step 60 ↓ |
|---|---|---|---|---|
| Base model (before unlearning) | 0.2842 | +0.0051 | +0.0051 | 27.8 (step 0) |
| Exact retrain from scratch (Gold, 30k steps) | 0.2791 | 0.0000 | 0.0000 | 27.7 |
| Retain-only fine-tuning (Naive) | 0.2855 | +0.0064 ± 0.0056 | +0.0056 ± 0.0053 | 25.4 (step 132) |
| Exact LFD-MC () | 0.2861 | +0.0070 ± 0.0033 | +0.0035 ± 0.0031 | 26.8 |
| Weighted LFD-MC () | 0.2815 | +0.0024 ± 0.0031 | −0.0073 ± 0.0022 | 26.6 |
| Weighted LFD-MC () | 0.2764 | −0.0027 ± 0.0028 | −0.0195 ± 0.0044 | 26.4 |
| Retain + Forget Ascent (128 retain + 128 forget, ) | 0.2738 | −0.0053 ± 0.0071 | −0.0175 ± 0.0022 | 28.9 |
Eq. (2) supplies the active repulsion needed for rapid erasure; Retain + Forget Ascent uses pre-split sets, while LFD matches it on unsplit .
Q3. (Why EBR stops short of 1.0% Gold) Without real retain images, classifier scores can only reweight the base generator's imperfect 65,536-sample synthetic bank : even a 0%-error Oracle filter (using the test evaluator itself to strip 100% of odd digits from the bank) leaves 9.2% generated odd digits at 300 steps, and 1,200-step EBR plateaus at 6.1%—demonstrating an inherent ceiling shared by all data-free methods that rely solely on classifier scores. (Proxy sensitivity & cross-dataset scope) Dividing proxy logits by 4 (making probabilities uncertain at 99.0% accuracy) degrades to 16.6%, whereas sharpening to cuts leakage to 6.7% (beating hard filtering's 9.9%); downsampling digits to blurry inputs (97.5% proxy accuracy, testing Proposition B.3's lossy view ) raises leakage only slightly to 11.2% (10.2% Hard). On 5-seed CIFAR-10, data-free EBR cuts airplanes from 9.41% (uniform bank, FID 16.94) to 5.67% (FID 15.87, ); conversely, on single-ID CelebA (), a 65,536-sample bank contains only 4 target-person images, starving any bank-reweighting method and confirming rare-instance removal requires LFD. (ContinualFlow) Simone et al. (2025) achieve 0.05% odd leakage using an unreleased convolutional autoencoder + latent flow (releasing only 2D toy code). Matching Appendix G, we will rewrite Section 7.2 and Table 2's caption as "Pixel-space ContinualFlow objective", clarifying that direct image-to-image interpolation stalls in pixel space without an autoencoder whereas EBR succeeds in pixel space inside one model.
| MNIST Odd-Digit Removal (65k synthetic bank) | Seeds | Steps | Odd Digits Left ↓ | Retain Feature FID ↓ |
|---|---|---|---|---|
| Base model (generates ~50% odd, ~50% even) | 1 | 0 | 49.3% | 357.5 |
| Exact retrain on real even digits (Gold) | 1 | 117.5k | 1.0% | 1.8 |
| Uniform bank fine-tuning () | 5 | 300 | 48.5 ± 3.7% | 377.5 ± 42.7 |
| EBR (, 99.0% proxy) | 5 | 300 | 10.7 ± 1.0% | 51.7 ± 15.3 |
| Hard filter (99.0% proxy) | 2 | 300 | 10.3% | 61.2 |
| Oracle classifier filter (0% error on bank) | 2 | 300 | 9.2% | 63.9 |
| EBR (, 99.0% proxy, longer budget) | 1 | 600 | 6.6% | 27.8 |
| Under-confident proxy (logits ), EBR () | 1 | 300 | 16.6% | 83.5 |
| Under-confident proxy (logits ), Hard filter | 1 | 300 | 9.9% | 60.2 |
| Under-confident proxy (logits ), EBR () | 1 | 300 | 6.7% | 73.5 |
| -downsampled proxy (97.5% acc.), EBR () | 1 | 300 | 11.2% | 65.6 |
| -downsampled proxy (97.5% acc.), Hard filter | 1 | 300 | 10.2% | 67.6 |
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns. However, three important questions remain:
1. Methodological novelty. The authors acknowledge that LFD shares its loss decomposition and importance-sampling mechanism with SISS, while EBR's energy-weighted objective follows existing EFM formulations. Although the velocity parameterization, prior correction, and single-model deployment provide useful distinctions, could the authors clarify what genuinely new algorithmic or theoretical contributions these methods introduce beyond adapting established techniques to the unlearning setting? In particular, how does Proposition B.3 go beyond standard Bayesian identities and conditional-expectation decomposition?
2. LFD's actual contribution and experimental attribution. The new ablations show that Exact LFD provides limited improvement over naive retain-only fine-tuning, while Weighted LFD achieves its gains primarily through the additional norm-controlled forget-ascent term, which resembles existing strategies in SISS. Moreover, Retain + Forget Ascent achieves comparable results. What evidence demonstrates that LFD's proposed decomposition, rather than the additional forget-ascent mechanism, is responsible for the reported improvements? Does Weighted LFD offer a distinct computational or optimization advantage over this simpler baseline?
3. EBR's theoretical claims and experimental evidence. Proposition B.3 establishes recovery of the retain-conditional distribution of the synthetic generator, which need not match the real-data retraining distribution. Furthermore, the oracle-filter experiment's 9.2% residual leakage after 300 updates does not establish an inherent limitation of classifier-only methods, as optimization and approximation errors remain possible. Could the authors clarify these theoretical distinctions and explain how the remaining gap to exact retraining should be interpreted? Given the acknowledged limitations of the pixel-space ContinualFlow reimplementation, what conclusions about EBR's comparative effectiveness remain supported?
I appreciate the additional evidence and would welcome clarification of these points before finalizing my assessment.
Rebuttal Part-1
We thank the reviewer for their meaningful feedback, specifically for highlighting our two-regime formulation and exact-retrain benchmarks, and for pointing us to EFM (Zhang et al., ICLR 2025).
Weakness 1: Novelty vs. SISS, EFM & ContinualFlow
We will cite EFM, present general Boltzmann-weighted flow matching (Eqs. 4 and 6) as background, and remove "global optimizer" from the Abstract so our genuine contributions stand out clearly. In particular, unlike ContinualFlow—which freezes the unedited retain+forget base generator (keeping forgotten data intact in its weights) and trains a second flow network on top (doubling test-time ODE sampling cost)—EBR overwrites the single base generator in place (1× sampling cost) using exact prior-corrected density ratios.
Q1. (LFD vs. SISS) SISS formulates importance-sampled unlearning for score-based diffusion SDEs; under linear Gaussian bridges xt=(1−t)x0+tz (Appendix E), LFD shares SISS's defensive-mixture weights while regressing deterministic ODE velocities utθ directly—inducing a ((1−t)/t)2 time weight relative to unweighted score matching without a score network or stochastic SDE sampler—and adds our O(πF) gradient-dilution diagnosis and ρ∝Nf size-scaling rule. (EBR vs. EFM & ContinualFlow) EFM assumes a known offline-RL reward R(z), whereas unlearning provides only an imperfect classifier r^(z) whose training prior differs from the deployment prior πR; Proposition B.3 proves that prior-shift odds correction at β=1 recovers the exact retain law q0(⋅∣R=1) and splits squared density-ratio error into irreducible feature insufficiency (E[Var(r(Z)∣S)]/πR2) plus calibration error, while updating one noise-to-data generator in place rather than ContinualFlow's heuristic σ(−γ⋅logit) two-model q0→q1 wrapper.
Each regime contributes distinct unlearning theory, scaling rules, and single-model deployment beyond SISS, EFM, and ContinualFlow.
Weakness 2: Exact LFD, Cross-Dataset Scope, Faces & MNIST Gap
We will streamline the presentation, separate Exact LFD (ρ=0) from Eq. (2)'s rescaled forget step in Section 5, and add five-seed CIFAR-10 class-removal experiments where both LFD and EBR succeed on the same base generator—alongside the exact sample-starvation boundary showing why rare identity deletion requires LFD.
Q2. (Exact LFD & attribution) On CelebA (
Gap = Method − Goldon target-identity FaceNet reconstruction cosine across 5 seeds, 256 states/step), the 180 forget images represent πF≈10−3 of training data, making LFD's raw forget gradient gF ~400× smaller in norm than its retain gradient gR despite aligning at cosine +0.981 with direct forget ascent. Consequently, Exact LFD (ρ=0) stalls like Naive retain-only fine-tuning, whereas Eq. (2) (rescaling gF to norm ρ∥gR∥2) drives >1,000× faster convergence to Gold: it matches a direct 128-retain + 128-forget split (Retain + Forget Ascent; paired 95% CIs span 0 across steps 12–60, seed-17 step-12 FID 30.1 vs. 31.5), centers single-identity runs (Nf=30) on Gold (−0.0007 vs. −0.0248 unscaled) via ρ(Nf)=ρ0(Nf/Nf,0), and on 5-seed CIFAR-10 airplane removal cuts airplanes from 9.11% (Base) and 5.67% (Naive) to 4.18% (p=0.031, retain FID 11.55 vs. 11.50). (Qualitative faces) Because OpenReview comments are text-only, we will add our paired 7-column CelebA reconstruction figure (Original,Base,Gold,Naive,Exact LFD,Retain + Forget Ascent,Weighted LFDfrom identical noise seeds) to the revised paper; visually,Weighted LFDerases target-identity traits to matchGoldwhile preserving pose, lighting, and non-deleted identities (step-12 retained cosine 0.2718 vs. 0.2730 forGold).