- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Machine unlearning removes knowledge associated with a forget set from a learned model. The unlearned knowledge can resurface through relearning attacks. Typical relearning attacks require fine-tuning the unlearned model, which leads to significant changes to the model parameters. The paper proposes a method, BFRA, to recover unlearned knowledge by flipping a small number of bits in the model parameters. They formulate the goal of identifying a small number of bits to flip that minimizes loss on the forget set while maintaining small loss on the retain set as a discrete optimization problem. They select candidate bits by solving a first-order approximation of the problem. Then they select the actual set of bit flips based on the actual realized loss reduction while maintaining the retain accuracy constraints. To make the method more efficient, the set of eligible bit flips is restricted to sign and exponent bits. In the experiments, restricting to this subset of bits was shown to be effective. High forget accuracy was achieved while maintaining retain loss by only flipping a small number of sign and exponent bits.
Critical strengths
With only a small number of bit flips, BFRA recovers a high forget accuracy while maintaining comparable retain accuracy to the original unlearned model. The empirical measurements of the runtime of BFRA demonstrate that solving the first-order approximation of the optimization problem is not too expensive. Restricting the bit search space to sign and exponent bits increases efficiency, and with this restricted search space, BFRA can still achieve high forget accuracy.
Critical weaknesses
In the main experiments, BFRA has access to the entire forget set, and then the evaluation metric is the accuracy on that forget set. This doesn't necessarily mean that BFRA has managed to recover unlearned "knowledge" that generalizes beyond the exact examples in the forget set. For example, for the TOFU experiments, could the authors evaluate the accuracy of the various methods on reworded or related versions of the questions in the forget set? There are some experiments where BFRA is only given a subset of the forget dataset, and then forget accuracy is evaluated on a held-out portion. I think these experiments should be expanded to demonstrate that BFRA can resurface forgotten knowledge, rather than just specific forgotten examples.
Since BFRA solves an approximation to the discrete optimization problem, there's no guarantee that the retain accuracy constraint will be satisfied. The procedure simply keeps flipping bits (increasing T) until the retain accuracy drops below a desired level. It needs a manual check to ensure that the retain accuracy is satisfied. It seems that this retain accuracy constraint and manual check could be added (or incorporated with a slight change in objective) to any standard bit-flipping attack.
Questions for authors
In the comparison with baseline bit-flip attack methods (Table 4), were the baseline methods also restricted to sign and exponent bits? In the ablations, the authors establish that the bit search space can significantly impact the effectiveness of the attack. It would be good to compare BFRA to other bit-flip attacks under the same bit search space to see how much of the improvement can be attributed to this restricted search space versus the optimization of BFRA?
The baseline bit-flip attacks target a different objective than BFRA. Can the baseline objectives be easily modified to target the same objective as BFRA? With the modified objective, how do these methods compare to BFRA?
See the weakness section for my concern on the distinction between recovering knowledge vs exact examples. How does fine-tuning compare to BFRA on recovering forgotten knowledge beyond the exact examples in the forget set? How does fine-tuning compare to BFRA when only a subset of the forget set is given?