- Rating
- 2
- Confidence
- 2
- Informativeness
- 2(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates the issue of untuned batch-norm statistics in the machine unlearning scenario.
Specifically: machine unlearning methods act on a model to remove the impact of training on a data subset "forgotten data". This procedure may leave the batch-normalization statistics "stale", i.e. that the layer-wise mean and variance buffers do not correspond to the relevant activation means and variances on the data remaining after the "forgotten data" is removed.
A simple well-known procedure to remedy this is to re-tune the buffers on the data that remains [1].
The paper investigates the impact of the buffer re-tuning on two public checkpoint releases (MU-Bench and Unlearning Comparitor), covering seven unlearning methods, plus two self-trained populatons. The method SalUn is excluded because the published forget accuracies could not be reproduced.
Re-tuning the batch-norm statistics does alter the accuracy on the different data partitions beyond seed noise, for roughly a fifth of the checkpoints tested. A handful of pass/fail verdicts (benchmark-specific rules for whether an unlearning was succssful or not), but method rankings are unchanged. Which checkpoints are affected, and why, is left unclear.
Consequently, the paper recommends that benchmarks and checkpoints are released with information about how the batch-norm statistics are treated.
[1] Y. Wu and J. Johnson. Rethinking "Batch" in BatchNorm. arXiv:2105.07576, 2021. https://arxiv.org/abs/2105.07576
Critical strengths
1. Careful reproduction of third-party checkpoints.
I appreciate the paper's effort to get a proper baselines before starting. The paper is a good demonstration of the challenge in reproducing other results in the literature.
Critical weaknesses
1. The paper does not establish why its findings matter, or what causes them.
The problem setting and setup is clear and well-explained. As soon as the paper turns to the Results section, reading and understanding the contributions becomes extremely challening and it is difficult as a reader to extract takeways (generally, or for unlearning specifically).
As far as I understand, the paper's own results suggest that benchmark conclusions largely survive refit: method rankings by forget accuracy are unchanged (lines 399–401), most flipped verdicts sit at the threshold (lines 372–374), and the authors describe the verdict result as "an existence result on four checkpoints" (line 465). The cause of the shift is left open ("no procedural cause identified", line 462), even though the abstract states a negative conclusion about removed data in the statistics.
Given the framing (provided in my summary of the paper) I would have expected a statement similar to: "For methods of class X, the forget, retain, and test accuracy were shifted (significantly) like A, leading to a systematic change in benchmark conclusions for benchmarks B. However for class Y no such change occurs. We investigate this, and find it is because [...]"
It is also fine if the results really are inconclusive. If so, this should be stated up-front, not only in the Limitations section.
This critical weakness 1. is problematic for me as a) it is already well-know in the literature that batch-norm statistic tuning can have a systematic effect [1,2], and b) the particular problem in batch-norm unlearning, specifically constrained to convolutional batch-norm models, whereas modern architectures typically use RMS/LayerNorm [3,4], and so the paper needs to make it clear why the problem matters.
[1] Y. Wu and J. Johnson. Rethinking "Batch" in BatchNorm. arXiv:2105.07576, 2021. https://arxiv.org/abs/2105.07576
[2] P. Izmailov, D. Podoprikhin, T. Garipov, D. Vetrov and A. G. Wilson. Averaging Weights Leads to Wider Optima and Better Generalization. UAI, 2018. https://arxiv.org/abs/1803.05407
[3] J. L. Ba, J. R. Kiros and G. E. Hinton. Layer Normalization. arXiv:1607.06450, 2016. https://arxiv.org/abs/1607.06450
[4] B. Zhang and R. Sennrich. Root Mean Square Layer Normalization. NeurIPS, 2019. https://arxiv.org/abs/1910.07467
Questions for authors
1. [Clarification] Based on your results, does anything an unlearning researcher cares about change after a refit?
Refitting appears to raise all of forget, retain and test accuracies, while benchmarks judge checkpoints on comparisons (e.g. the gap between forget and test accuracy, or method rankings).
How often does the gap itself move beyond some threshold? Does any method family show a consistent change in benchmark conclusions?
A systematic change for some class of methods would directly address my main concern (Critical Weakness 1) and could change my assessment.
2. [Clarification] What do you think is causing the shift?
The paper names two possible explanations: removed data surviving in the shipped statistics ("deletion failure"), or statistics that are stale relative to the final weights (lines 053, 077). The abstract rules out the first (lines 021–022), based on the targeted removed-image test in Section 4.2: at the final, fixed weights, the statistics are refit with and without removed images swapped in, and the accuracies barely change (lines 311–316, Table 2).
I am not sure this test can rule deletion failure out. The shipped statistics of some methods were updated on batches made only of removed images while unlearning was running (lines 308–309), i.e. at earlier weights, whereas the test only adds removed images at the final weights. The test is also run only on MU-Bench (lines 306–307), not on Unlearning Comparator's class deletion (lines 206–207), where removed data should be easiest to detect.
Could you either show that the test would detect removed data in statistics accumulated during unlearning, or narrow the claim in the abstract? If the cause is stale statistics, please say so explicitly. If removed data does survive in the statistics, that would be a genuine unlearning finding and could change my assessment.
3 comments
Response to Reviewer BN7a (2/2): cause of the shift, new class-deletion control
Q2 — what causes the shift, and can the Section 4.2 test rule out deletion failure? You are right that it cannot, and your suggestion led to a result we did not have. We ran the test on Unlearning Comparator's class deletion (new Appendix G, Table 14; 6 checkpoints × 10 deleted classes × 3 draws). At fixed weights we add images of the deleted class to the refit and compare two directions in the space of running means: the one in which the class moves the fitted statistics, and the one in which the shipped statistics differ from a retained-only fit. A projection of 1 means the shipped statistics differ from a retained-only fit exactly as a fit containing the class at its 10% share does.
| checkpoint (10 deleted classes each) | cosine with deleted-class direction | classes > 0 | projection (median) | refit shift, retain test (pp) |
|---|---|---|---|---|
| original model (positive control) | +0.93 | 10 of 10 | 0.98 | −0.75 |
| retrained reference | −0.06 | 3 of 10 | −0.07 | +0.02 |
| CF-k | +0.03 | 7 of 10 | 0.02 | +0.05 |
| AdvNegGrad | +0.42 | 10 of 10 | 7.8 | +1.89 |
| SalUn | +0.01 | 7 of 10 | 0.31 | +2.22 |
| SSD | +0.37 | 9 of 10 | 11.9 | +11.36 |
- The test detects removed data when it is there. The original model's statistics were fitted with the class present, and the test recovers it (cosine +0.93, projection 0.98, draw-to-draw sd 0.006). CF-k has the same batch-norm inputs as the original model but statistics refreshed on retained data, and shows nothing.
- Removed data does survive in the shipped statistics of two of four methods. AdvNegGrad is aligned on all ten classes and every draw, about eight times as far along the class direction as the class at its share would place it, as statistics averaged largely over deleted-class batches would be. SSD is aligned on nine classes; its code takes no training-mode pass, so it ships pre-unlearning statistics that contain the class by construction.
- It is not the whole story. The class direction carries about a fifth of the squared offset for both methods, and SalUn shifts as far under the refit (+2.22 pp) with no trace of the class. With the class at its share in the fit, retain accuracy moves by at most 0.13 pp and forget accuracy stays at 0% (all but SalUn).
So both explanations occur: stale statistics, and statistics that carry removed data. We do not separate how much of a shift each causes, and we no longer claim that removed data "does not explain" the movement. The abstract now reads: "Adding removed images to the refit at the final weights does not reproduce the shift on MU-Bench, where the removed data is a random subset. On class deletion, a control that recovers the deleted class in the original model's statistics finds it in the shipped statistics of two of four methods." The same narrowing is made in the introduction, Section 4.2, the limitations and the conclusion. The control reads a direction at the final weights, on one release; SalUn's ten checkpoints are included although only two reproduce their published numbers.
All changes in the revised PDF are in blue.
Response To Authors
Thank you for your detailed responses and running the additional experiments.
Q1. Based on your results, does anything an unlearning researcher cares about change after a refit?
Thank you for running this experiment. If I understand correctly, the conclusion is:
- refitting batch-norm statistics moves individual checkpoint numbers
- benchmark conclusions remain unchanged (rankings are unchanged, and the gap moves on 6 of 221 checkpoints, which does not survive the false discovery rate control). Despite this somewhat "negative" result, I do think "does tuning batch-norm change anything?" is a fair question, and the paper now gives a clear answer. I appreciate that this is now stated up front.
My remaining concern here is scope - the main counts come from one benchmark, on CIFAR with ResNets. I understand the authors discuss this in the limitations section.
Q2. What do you think is causing the shift?
Thank you for the class-deletion control experiment, and for narrowing the claims of the paper.
As I understand it, the control refits each checkpoint's statistics with and without images of the deleted class, and checks whether the shipped statistics differ from the class-free refit in the same way that adding the class does [L1228–1239]. The control works when the weights are unchanged: it finds the deleted class in the original model's statistics [L1239–1241]. My question was about the harder case, where the statistics are collected while unlearning changes the weights (e.g. AdvNegGrad). There, I am not yet convinced the control can tell "the class is in the statistics" apart from "the weights were changed to forget the class". For a few reasons:
- The control does not look for the class directly, only for this matching pattern. As far as I understand, unlearning a class tends to change the weights most where the network responds to that class, so out-of-date statistics could be off in those same places with no class data in them.
- The control's numbers depend strongly on the weights. SSD ships its pre-unlearning statistics, which contain the class by construction [L1271–1272], yet its projection is 11.92 against 0.98 for the original model (Table 14).
- SalUn shows no match even though its statistics are updated on removed-only batches [L499–500], so the readout under moving weights does not seem well understood either way.
As a simple check, give SSD's weights statistics known to contain no class data (e.g. computed at the original weights on retained data only) and rerun the test: if the match remains, it comes from the weights; if it disappears, from the removed data. For AdvNegGrad, the equivalent is rerunning it with statistics updated only on retained data.
Until then, I would move the SSD caveat [L1271–1272] from Appendix G into the main text, next to the "two of four methods" claim.
Score
I am raising Informativeness from 1 to 2: the paper now gives a clear answer to a fair question, I learned something useful from it. I am keeping my Rating at 2. In my review I named what would change my assessment:
- a systematic change in benchmark conclusions (Q1), or
- removed data surviving in the statistics (Q2). The answer to Q1 is no, and Q2 is not yet established, as far as I can tell. If the checks above show that removed data does survive in the shipped statistics, that would be the genuine unlearning finding I asked about, and I would raise my score.
I thank the authors again for their responses during this rebuttal.
Response to Reviewer BN7a (1/2): what changes after a refit
Thank you for the careful reading. We agree with both main points and answer them directly.
Q1 / W1 — does anything an unlearning researcher cares about change? Mostly no, and we will say so up front instead of in the Limitations.
We computed what you ask for from the per-checkpoint results already in the supplement: the shift of the signed gap d_f − d_t under the refit, decided by the same rule as the paper (all ten refits beyond the 1.2 pp margin).
The gap moves past the margin on 6 of 221 checkpoints, against 47 for forget accuracy alone. Across the 221, the forget and test shifts correlate at r = 0.97, and among the 47 displaced checkpoints the median gap shift is 0.34 pp. So most of the movement is common to all three accuracies, as you suspected. With a two-sided sign test the 6 do not survive the false-discovery control we use elsewhere, so we report them as a count only. All 6 move the same way (forget accuracy rises more than test accuracy), 5 are bad teaching, and 4 are the CIFAR-10 ResNet-50 bad-teaching checkpoints the refit raises by +11 to +20 pp.
No method family shows a consistent change in benchmark conclusions: NegGrad's mean gap shift is within 0.1 pp of zero in every group, rankings by forget accuracy are unchanged, and 12 of the 16 flipped verdicts are threshold effects. What does change is narrower: individual published rows (47 of 221), and 4 verdicts.
We will add this table, and restate the contribution in the abstract and introduction as what it is: a reproducibility result about per-checkpoint numbers in released tables, with benchmark-level conclusions largely intact.
Why it still matters, given [1, 2] and the move to Layer/RMSNorm. We cite both [1] and [2] and use the estimator of [1]; we do not claim that recomputing statistics changing accuracy is new. What was open is whether tables the field already cites for third-party checkpoints survive it, row by row. The scope is batch-normalized classifiers only, as stated in Limitations. That scope is still where audits against a retrained reference are run: both public releases with per-checkpoint tables are BN ResNets, and 19 of the 42 papers in our rule-based literature sample evaluate BN convolutional models against a retrained reference, of which 4 mention the statistics.