acceptodds
Under review as a conference paper at ICLR 2027

FF-Erase: Machine Unlearning and Verification for Forward-Forward Models

Abstract

The Forward-Forward (FF) algorithms provide a promising alternative to backpropagation (BP) by training neural networks through independent layer-wise goodness objectives. However, machine unlearning for FF models remains largely unexplored, which is a fundamental model function aiming to remove the influence of designated training samples from a trained model while preserving its utility on the remaining data. Directly adapting BP-oriented unlearning methods either causes ineffective unlearning or induces model collapse. This is because of two fundamental challenges specific to FF unlearning: (\romannumeral 1) each FF layer lacks a compatible unlearning target with the other layers (i.e., coordinated yet layer-specific directions), and (\romannumeral 2) the under/over-forgetting balance is independent across layers and thereby difficult to determine. To address these challenges, we propose FF-Erase, the first FF-specific machine unlearning framework. FF-Erase leverages a low-cost guidance model to instantiate a stable and structurally realizable goodness space. This establishes coordinated yet layer-specific anchors for precise goodness realignment, thereby avoiding the optimization collapse of independent layers. Such a goodness anchor also enables layer-independent unlearning adaptation, allowing FF-Erase to adjust the under/over-forgetting balance layer-wise. We further propose goodness-based membership inference (G-MIA), an FF-specific unlearning auditing tool that efficiently and accurately verifies the removal effectiveness for an FF unlearning algorithm. Extensive experiments across architectures, datasets, forgetting ratios, and unlearning scenarios show that FF-Erase consistently forgets target data effectively while preserving retained-data utility and avoiding model collapse. FF-Erase achieves up to speedup over retraining, establishing an effective and efficient foundation for machine unlearning on FF models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Forward-forward learning, Machine unlearning, Data removal

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Y5syOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work explores machine unlearning using forward-forward (FF) algorithms as an alternative to the predominant back-propagation (BP) unlearning literature. This paper discusses and resolves two FF-specific challenges: (i) layers lack coordinated yet layer-specific unlearning targets, and (ii) the balance between under- and over-forgetting differs across layers. The paper also proposes G-MIA: an MLP membership classifier trained on temperature-scaled layer-wise goodness vectors, to help better evaluate FF unlearning quality.

Critical strengths
  1. Clear diagnosis of a concrete failure mode. The layer-wise CKA analysis (Tab. 26) shows that output-only unlearning leaves FF layers 1–12 essentially unchanged, even though the predictor still relies on those layers. It also shows that naive per-layer extensions over-perturb every layer. This is a useful caution for any model trained with layer-local objectives.

  2. The audit is well validated. Table 14 reports AUCs for retained vs. unseen and retained vs. forgotten samples. This rules out the trivial explanation that the frozen attacker simply fails across the board after unlearning. Table 12 cross-checks G-MIA against eight other membership inference attacks (MIAs).

Critical weaknesses
  1. The case for FF unlearning is not made. FF is still a research-stage training method with accuracy gap to BP. The paper’s own Tab. 27 reports 82.2% vs. 87.0% test accuracy for VGG-13/CIFAR-10. I am not aware of FF-trained models in deployment. The paper motivates the problem with edge devices, yet every experiment runs on A800 GPUs, and Appendix I states that the memory measurements do not reflect edge hardware. There can be scenarios for unlearning where BP isn't available either, but the authors should discuss and test on those settings. FF-Erase also needs access to the full retained set and a held-out validation set, which weakens the on-device argument.

  2. The technical novelty over existing unlearning methods is limited. Challenge (i) is the problem that reference-guided and teacher-based unlearning already address. ReGUn, which the paper uses as a baseline, already distills forget samples toward a reference distribution built from disjoint held-out data. Bad Teacher distills forget samples from an incompetent, randomly initialized teacher. FF goodness vectors act as per-layer class-logit heads. FF-Erase is therefore essentially reference-model distillation applied at every head, with per-sample targets from a model trained on held-out data. Challenge (ii) is handled by gradient-norm balancing in the style of GradNorm and MetaBalance. The paper should clarify what is fundamentally FF-specific beyond applying these components layer by layer.

  3. The claim that BP methods are “incompatible” with FF is not well tested. The paper considers only two adaptations: a. “direct” adaptation, which according to Table 26 changes only the final layer; b. “layer-wise” adaptation, with inputs detached between layers. End-to-end BP unlearning through the whole FF network, on the aggregated predictor or on the sum of layer losses, is missing. The baselines also appear to be at a disadvantage:All methods share one stopping rule, based on relative improvement of the local objective. This rule does not stop gradient-ascent methods before collapse. Standard practice is to stop when forget-set accuracy approaches held-out accuracy. Layer-wise SCRUB has an explicit retain-set term, yet it collapses to exactly chance level in every setting. Learning-rate sweeps are given only for GA and ReGUn. Standard baselines are absent: fine-tuning on the retained data alone, NegGrad+, SalUn, and the training-free SSD.

  4. Several reported statistics raise reproducibility concerns. In Table 6 (VGG-13/CIFAR-100), the relative standard deviations (RSDs) equal those in Table 3 (VGG-13/CIFAR-10) plus exactly 0.1 in essentially every cell. The same +0.1 relation holds between Table 7 (MNIST) and Table 8 (Fashion-MNIST) in 54 of 57 cells. It also holds between Table 6 and Table 9 (ResNet-50) for all rows other than the layer-wise ones. With five seeds, dispersions measured independently on different datasets and architectures would not line up this way? The MNIST shadow model reportedly takes 982.7s to train, while the MNIST target model takes 436.6s (Table 7). Training the Cross-Domain guidance model on about 5k CIFAR-100 images takes 953.7 s, compared with 57.1 s for Val-Train on 5k CIFAR-10 images...Taken together, these make it hard to tell whether the reported variability and costs were measured as described at this point.

Questions for authors

Please refer to Critical Weaknesses section. Also 1. How does FF-Erase compare with end-to-end BP-based unlearning (e.g., SCRUB, ReGUn, SalUn) applied through all layers of the FF network? In what concrete deployment setting is backpropagation unavailable at unlearning time? 2. What is the guidance model’s test accuracy at each validation split (1–10%)? How does FF-Erase compare with layer-wise ReGUn when ReGUn is given the same per-sample guidance targets?

Reviewer_v17mOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims to solve the unlearning problem for forward-forward models and identifies two fundamental problems: 1) FF layers lack compatible unlearning targets; 2) the under/over-forgetting balance is independent w.r.t. layers. The authors propose FF-Erase, which mainly uses goodness as a training and verification mechanism.

Critical strengths
  1. This paper aims to address a new problem, unlearning in FF models.
  2. This paper points out 2 key issues of the above problem, which is intuively easy to understand.
Critical weaknesses
  1. The presentation can be improved. For instance, the goodness value is introduced on page 3, while its mathematical formulation appears only on page 4.
  2. The proposed method does not differ fundamentally from conventional machine unlearning approaches. Both rely on constructing anchors for distribution alignment; the main change here is alignment over goodness values. This limits the methodological novelty of the work.
  3. The discussion on the under/over-forgetting balance (the second issue raised in the paper) builds on existing, somewhat outdated strategies. Recent literature has extensively investigated this trade-off, e.g., [1].
  4. G-MIA appears to be merely a local formulation adapted for FF models following standard MIA pipelines. It is unclear what unique challenges membership inference faces specifically within FF architectures; the differences seem largely formal.
  5. While the paper frames the problem nicely, I am not convinced that FF unlearning poses sufficiently distinct, substantial challenges. As a result, the motivation appears somewhat overclaimed. Methodologically, the work mostly adapts existing machine unlearning techniques to perform layer-wise optimization under the FF paradigm. The contribution is primarily formal, and I do not identify new conceptual insights for the field.

[1] Unlearning concepts in diffusion model via concept domain correction and concept preserving gradient. AAAI 2025.

Questions for authors

see the weakness

Reviewer_NaPcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces FF-Erase, a machine unlearning algorithm for FF networks, using layer-specific goodness anchors. the paper also prosed G-MIA, a goodness-based membership-inference auditing method for verifying unlearning effectiveness.

Critical strengths

The FF networks are relatively new model and architecture that has never been explored previously and this research papers attempts to first identify the challenges with unlearning of these models and then proposes the unlearning algorithm for these models. Studying these models and the challenges of the unlearning is the novel challenge that this paper tried to address.

Critical weaknesses

The authors usually use reference to the notions that are not properly described and formulates before referencing them. It makes the following the paper very difficult, as the reader has to either guess or look for the notion in the paper. the problem formulation and research statement is not clearly stated. Even the idea and research are interesting, the disorganized and poorly written paper makes it difficult to follow and read the algorithm.

line 46,47,48) revise the intro. you are talking about the FF and suddenly shift tot he edge devices and unlearning. you need to state if that the edge devices are using the FF models then the connection becomes clear

Figure 1) it is better for the figures to the be placed on the top of page.

Preliminaries ) this section needs revision. it seems that at first you talk about the FF, then you provide a bit of information about the algorithm, and then you suddenly jump in more depth. so please revise this to better explain the algorithm

line 178 ) above is wildly →\rightarrow reference figure

line 256) g∗lg^{l}_{*} is not introduced. from my understanding it could be the goodness vector for the y label.

formula 3) this formula attempts to minimize the kl divergence between the sofrmax of the g for each layer. so the formula attempts to minimize the kl divergence, between the goodness of forgetting data and unlearning model

line 230) "a goodness distribution valid" →\rightarrow what do you mean by validity?

line 244) what are the undergforgetting and over forgetting layers and how do you identify it.

formula 5) what is g-tilda? you didn't introduce it!

Questions for authors

see the weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.