- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Feature Reflection (FR), a retain-data-free unlearning method that uses only the pretrained model and forget data. FR reflects the current model’s features around their pretrained counterparts, adds multiplicative Gaussian noise, and minimizes the classification loss on the resulting features. The authors show that, under a fixed linear head, reducing the clean reflected-feature loss raises a lower bound on the original forget loss, while a second-order analysis interprets the noise as a curvature-weighted regularizer on feature deviation from the pretrained model. FR is evaluated on CLIP across eight vision datasets under full-data, few-shot, and long-tailed settings, and on Zephyr-7B-β using WMDP and MMLU. The reported results show a stronger forgetting–retention trade-off than the evaluated baselines, supported by ablations on reflection depth, noise level, regularization variants, and training dynamics.
Critical strengths
- Simple and practically useful retain-free formulation. Feature Reflection converts forgetting into ordinary loss minimization on reflected intermediate features, requiring only the pretrained model and forget data. This is computationally straightforward and addresses an important setting in which retain data are unavailable for parameter updates.
- Broad and generally consistent empirical validation. The method is evaluated across multiple CLIP architectures, eight vision datasets, several data regimes, and a language-model unlearning benchmark. The reported results and ablations consistently suggest a favorable forgetting–retention trade-off, indicating that the core mechanism is not limited to a single dataset or model family.
Critical weaknesses
- FR’s novelty over task-vector negation is limited. Clean FR is an online, feature-space version of fine-tune-and-negate, and the two procedures are equivalent for a linear or tangent model. Reusing the same Jacobian removes TA’s cross-model Jacobian term, but it does not guarantee that the update increases the original forget loss because and can differ. The paper’s clean analysis also does not directly apply to the noisy objective used in the experiments. The theoretical distinction from task-vector negation is therefore narrower than the paper suggests.
- The visual evaluation is unnecessarily limited to CLIP. The main theoretical requirement is a fixed linear classification head, which can also be satisfied by freezing the head of a standard supervised model. The paper could therefore have evaluated FR on conventional ResNet or ViT classifiers and standard class- or sample-unlearning benchmarks. Since all vision results come from the CLIP task-arithmetic setting, it is unclear whether the gains transfer to other models and unlearning settings. The baseline comparison is also incomplete: the paper omits linearized variants of NegMerge and Task Simplex, as well as recent retain-free methods such as POUR and DELETE. These omissions make it difficult to assess either the method’s generality or its advantage over prior work.
Questions for authors
Clarification requests:
- Under a linear or tangent approximation, , clean FR appears to be equivalent, step by step, to fine-tuning on and then negating the resulting task vector. Is this interpretation correct? If not, which term makes the two procedures different?
- Equation 7 replaces TA’s cross-model Jacobian product with , but the alignment is , which need not be nonnegative when . What guarantee does FR provide on the direction of an individual update? How does this argument extend to the noisy objective actually used in the experiments?
- The method is described as retain-free, but retain validation data are used to select the scaling coefficient and hyperparameters. How would these choices be made when no retain data are available at all, and how sensitive are the reported results to this model-selection procedure?
Evidence needed for the current claims:
- Can the authors evaluate FR on a conventional supervised model, such as a ResNet or ViT with a frozen linear head, using a standard class- or sample-deletion benchmark and a retrained-without- reference? The theory is not specific to CLIP, so this experiment would help determine whether the reported gains transfer beyond the CLIP task-arithmetic setting.
- Can the authors compare against linearized variants of NegMerge and Task Simplex, and against recent retain-free methods such as POUR and DELETE on a common benchmark? These are close comparisons needed to assess FR’s advantage over prior work.