- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
This paper studies privacy leakage in Federated Unlearning, asking whether an ordinary client can infer another client's forgotten labels without accessing private data or individual model updates. To address this, the authors propose Active Unlearning Probing (AUP), which builds class-specific unlearning templates from the attacker's own data and matches them to changes in the global model's classifier layer using cosine similarity. Experiments across datasets, model architectures, unlearning mechanisms, and federation sizes assess the attack's effectiveness, robustness, and scalability. Results show that AUP achieves high forgotten-label inference accuracy over the tested baselines, and its effectiveness generally improves as the number of clients increases.
Critical strengths
- The paper identifies an important privacy risk without requiring access to the victim’s individual updates.
- The experiments provide strong initial evidence across several datasets and show the value of focusing on the classifier layer.
Critical weaknesses
Major Weaknesses
- My main concern is about the authors' claim that the victim’s signal remains fixed while background noise decreases as the number of clients grows. However, keeping the number of participating clients per round fixed does not guarantee that the victim’s update stays the same.
- The reported accuracy measures label identification only for detected events. The paper does not provide equally detailed results for missed events, false alarms, or the full detection-and-identification process. It also omits simple comparison methods with the same information access, such as classifier-bias ranking, one-step gradient matching, and class-wise loss-change scoring on the attacker’s own data.
Minor Weaknesses
- The main experiments assume IID data and the same label set across clients (Section 3). However, real federated learning systems often contain significant differences in client data distributions.
- Section 3 assumes that the victim participates continuously during a multi-round unlearning campaign, while other clients are sampled normally. However, the paper does not evaluate situations where the victim participates discontinuously, where its unlearning stream is shorter, or where multiple clients request unlearning at overlapping times.
Questions for authors
- Which algorithm was implemented? Did the reported experiments use the additional template-mean subtraction shown in Figure 1, or the version without this step in Algorithm 1 and Equation (8)?
- For each setting, how many forget events were attempted, detected, and correctly identified? Please report detection recall, false-positive rates on blocks without unlearning, and the fraction of all events that were both detected and correctly identified.
- What justifies neglecting prediction–feature covariance in Proposition 1? What assumptions control alignment over multiple ascent steps and rounds?
- How does AUP compare with classifier-bias-only ranking, one-step class-gradient matching, and normalized class-wise loss-change scoring using the attacker’s own data?
7 comments
Reply to Reviewer zuPt [Part 2/5]
We report both a threshold-free measure of separability and the operating points that different detection constants realize. Magnitude is the detector of Algorithm 3 applied to . Product applies the same threshold rule to . Each operating-point cell is / FPR / .
| Setting | Score | AUC | |||
|---|---|---|---|---|---|
| CIFAR-10 / ResNet-18 | magnitude | 0.999 | 1.000 / 0.050 / 1.000 | 0.983 / 0.017 / 0.983 | 0.867 / 0.000 / 0.867 |
| product | 0.997 | 1.000 / 0.150 / 1.000 | 1.000 / 0.087 / 1.000 | 0.983 / 0.042 / 0.983 | |
| CIFAR-10 / SGA-EWC | magnitude | 0.998 | 1.000 / 0.042 / 0.983 | 0.983 / 0.013 / 0.967 | 0.867 / 0.004 / 0.850 |
| product | 0.998 | 1.000 / 0.150 / 0.983 | 1.000 / 0.096 / 0.983 | 1.000 / 0.050 / 0.983 | |
| CIFAR-10, | magnitude | 0.696 | 0.100 / 0.017 / 0.067 | 0.033 / 0.004 / 0.033 | 0.000 / 0.000 / 0.000 |
| product | 0.941 | 0.483 / 0.029 / 0.450 | 0.250 / 0.017 / 0.233 | 0.117 / 0.008 / 0.117 | |
| CIFAR-100 / ViT-B/16 | magnitude | 0.995 | 1.000 / 0.204 / 1.000 | 1.000 / 0.093 / 1.000 | 0.778 / 0.000 / 0.778 |
| product | 0.998 | 1.000 / 0.519 / 1.000 | 1.000 / 0.352 / 1.000 | 1.000 / 0.241 / 1.000 | |
| Yahoo Answers / MiniLM | magnitude | 1.000 | 1.000 / 0.963 / 1.000 | 1.000 / 0.907 / 1.000 | 1.000 / 0.870 / 1.000 |
| product | 1.000 | 1.000 / 0.907 / 1.000 | 1.000 / 0.907 / 1.000 | 1.000 / 0.889 / 1.000 | |
| CIFAR-10, | magnitude | 0.942 | 1.000 / 0.467 / 0.550 | 1.000 / 0.400 / 0.550 | 1.000 / 0.389 / 0.550 |
| product | 0.961 | 1.000 / 0.456 / 0.550 | 1.000 / 0.411 / 0.550 | 1.000 / 0.389 / 0.550 |
The template-product AUC is between and in every monitoring setting, and detection performance depends on both score separability and threshold placement. Yahoo Answers has an AUC of while realizes an FPR of . The cause is where a threshold fixed on the calibration blocks falls relative to each setting's ambient scale. The mean over event-free monitoring blocks, first half and second half, against the calibration mean that sets the threshold, is and against on CIFAR-10 (), and against on CIFAR-100 (), and and against on Yahoo Answers (). On Yahoo Answers the ambient scale is 2.7 to 3.5 times the calibration mean, so the threshold sits below almost every event-free block. The detection constant is a setting-specific choice, and Section 5.2 reports the FPR next to every joint success rate.
At the magnitude score has AUC and its threshold fires on no block, while the template product has AUC and reaches at FPR for . Incorporating class direction therefore improves event detection in this configuration.
Reply to Reviewer zuPt [Part 3/5]
W2 & Q4. Comparison with classifier-bias ranking, one-step class-gradient matching, and class-wise loss-change scoring
A3. We ran all three under the attacker's exact information access. Bias ranking orders classes by and row-norm ranking by . Neither needs any probing. One-step gradient matching compares with the vectorized classifier-layer gradient of the loss on the attacker's own class- samples. Loss and accuracy scoring rank classes by the change in the attacker's own per-class loss and accuracy. For the normalized form, we report three variants of the loss-change decoder below: the raw rise , the relative rise , and the calibration-standardized rise , where and are the mean and standard deviation of the attacker's class- loss over the calibration blocks. The table reports the raw form, and the normalized forms follow.
| Setting | Bias | Row norm | One-step grad | Loss | Accuracy | AUP |
|---|---|---|---|---|---|---|
| CIFAR-10 / ResNet-18 | 1.000 | 0.883 | 1.000 | 1.000 | 1.000 | 1.000 |
| CIFAR-10 / SGA-EWC | 1.000 | 0.900 | 0.983 | 1.000 | 1.000 | 0.983 |
| CIFAR-10, | 0.867 | 0.533 | 0.900 | 0.883 | 0.950 | 0.933 |
| CIFAR-100 / ViT-B/16 | 1.000 | 1.000 | 1.000 | 0.806 | 0.667 | 1.000 |
| Yahoo Answers / MiniLM | 1.000 | 1.000 | 1.000 | 0.861 | 0.722 | 1.000 |
| CIFAR-10, long-tailed (imb. 100) | 1.000 | 0.917 | 0.917 | 0.444 | 0.389 | 1.000 |
| CIFAR-10, | 0.800 | 0.067 | 0.600 | 0.783 | 0.767 | 0.550 |
| Dirichlet | 0.278 | 0.250 | 0.306 | 0.250 | 0.194 | 0.333 |
Without a defense, bias ranking matches AUP at on CIFAR-10, CIFAR-100, Yahoo Answers, and the long-tailed partition, and it requires no probing. On multi-class forgetting, bias, loss-change, and accuracy-change scoring reach , , and , compared with for AUP. Under SGA-EWC these three decoders reach compared with for AUP. At , accuracy-change scoring reaches compared with for AUP. Under Dirichlet , AUP reaches and one-step gradient matching . Section 5.2 states the comparison in these terms.
Normalizing the loss-change decoder helps it, and we report where it then overtakes AUP:
| Setting | B3 raw | B3 relative | B3 standardized | AUP |
|---|---|---|---|---|
| CIFAR-10 / ResNet-18 | 1.000 | 1.000 | 1.000 | 1.000 |
| CIFAR-10 / SGA-EWC | 1.000 | 1.000 | 1.000 | 0.983 |
| CIFAR-10, | 0.883 | 0.983 | 0.950 | 0.933 |
| CIFAR-100 / ViT-B/16 | 0.806 | 0.889 | 0.972 | 1.000 |
| Yahoo Answers / MiniLM | 0.861 | 0.944 | 0.750 | 1.000 |
| CIFAR-10, long-tailed (imb. 100) | 0.444 | 0.583 | 0.556 | 1.000 |
| CIFAR-10, | 0.783 | 0.650 | 0.517 | 0.550 |
| Dirichlet | 0.250 | 0.278 | 0.333 | 0.333 |
Relative normalization raises the loss-change decoder from to at , above AUP's .
The comparison changes once the broadcast is perturbed, the defense that Section 6 proposes and Section 5.2 quantifies. Adding Gaussian noise of standard deviation to the broadcast classifier layer on CIFAR-100 / ViT-B/16, with pre-unlearning test accuracy in the first column:
| Model utility | Bias | Row norm | Loss change | Accuracy change | One-step grad | AUP | |
|---|---|---|---|---|---|---|---|
| 0 | 0.895 | 1.000 | 1.000 | 0.806 | 0.667 | 1.000 | 1.000 |
| 0.001 | 0.897 | 0.000 | 0.028 | 0.806 | 0.639 | 1.000 | 1.000 |
| 0.01 | 0.797 | 0.000 | 0.083 | 0.389 | 0.278 | 1.000 | 1.000 |
| 0.02 | 0.691 | 0.000 | 0.056 | 0.083 | 0.250 | 0.583 | 0.583 |
At , test accuracy is versus without noise. Bias ranking falls to and row-norm ranking to , while AUP and one-step gradient matching retain identification accuracy. At the two template-matching methods retain , where test accuracy decreases by percentage points and loss-change and accuracy-change scoring have fallen to and . Both template matchers reach at , where the decrease is percentage points.
Normalization does not change this. Under the same noise sweep on CIFAR-100 / ViT-B/16, the loss-change decoder reaches (raw / relative / standardized) at , and at , where AUP and one-step gradient matching are both still at .
Reply to Reviewer zuPt [Part 1/5]
We thank the reviewer for the detailed review and for the concrete requests for baselines and detection statistics. Revisions are highlighted in red in the revised PDF. Additional results are in Appendix B, with the alignment analysis in Appendix C.
A1. Holding clients per round fixed does not hold the victim's update fixed, and the claim that the attack strengthens with federation size has been withdrawn.
We measured the quantities that enter the signal-to-noise ratio. On CIFAR-10, five seeds, twelve forget events per seed:
Under fixed-total-data partitioning the victim's update norm ranges from 0.0010 to 0.3661, more than two orders of magnitude, so it is not a fixed quantity, and the background noise is not monotone in C under either protocol.
The decisive comparison is the controlled one at C=10, where the client count is held fixed and only the per-client data volume changes: the per-class loss on the forget set before unlearning rises from 3.0×10−5 to 7.5×10−4, the victim's local update norm rises from 0.0010 to 0.1090, about a hundredfold, and identification moves from 0.417 to 0.983. With a fixed per-client shard, identification across C=10,30,50,100 is 0.983,0.900,0.967,0.900, a range of 0.083 and not monotone.
The mechanism is the fitting state of the forgotten class. Gradient-ascent unlearning moves the classifier row in proportion to (p−y). Under the fixed-total-data partition, a smaller federation gives each client more data, the model fits the forget class to near-zero loss, the ascent gradient nearly vanishes, and the victim's forgetting leaves almost no trace in the parameters.
The paper now states that attack effectiveness follows the magnitude of the forgetting update relative to the background noise, both of which the partition protocol and the fitting state determine (Section 5.4). Proposition 2 is stated as a sufficient condition at a fixed alignment.
A2. Every stage is reported for the six monitoring settings, with block counts. Definitions are in Section 3: Rdet=TP/(TP+FN), FPR=FP/(FP+TN) on blocks without unlearning, Acond on detected events, and ASRjoint=Rdet×Acond over all attempted events. The CIFAR-10 settings use 60 events and 240 event-free blocks pooled over seeds, the multi-class setting 60 and 90, and the remaining settings 36 and 54. At k=3, the attempted, detected, and correctly identified event counts are: