acceptodds
Under review as a conference paper at ICLR 2027

When Unlearning Is Not the End: Controlled Oracle Recovery Against Federated Unlearning

Abstract

Federated Unlearning (FU) aims to remove the influence of client data on a shared large language model upon request. However, when the target model remains trainable, subsequent updates may restore the behavior intended to be unlearned. This creates an integrity threat: a malicious participating client may counter another client's deletion request by steering the shared model back toward its pre-unlearning behavior. Existing evaluations largely focus on the immediate post-unlearning state and largely overlook the possibility of such a recovery case. We introduce Controlled Oracle Recovery, an anti-unlearning audit that evaluates whether a deletion event reveals its target records and whether small target-guided updates can reverse forgetting more effectively than compute-matched control updates. Deletion target inference reaches at least \(0.999\) ROC AUC in 10 of 24 evaluations. After 100 local update steps, recovery using the deletion targets exceeds the strongest matched control in all 48 comparisons. In multiple settings, recovery also restores forgotten-answer probability to its pre-unlearning level while retaining at least \(95%\) of the utility measured before unlearning. The results reveal a broad but heterogeneous anti-unlearning susceptibility and establish a necessary local condition for a FU rollback attack on LLMs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
machine unlearning, federated learning, membership inference, large language models, privacy

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_XZjaOpenReview ↗
Rating
1
Confidence
3(−1)
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether a malicious client in federated learning can undo another client's deletion request by training the unlearned model further. The authors propose Controlled Oracle Recovery, an audit with two parts. The first part infers which records were deleted from the loss change between the pre- and post-unlearning models. The second part gives the attacker the exact deleted records and checks whether training on them restores the forgotten behavior more than compute-matched training on other data. Experiments use TOFU with Llama-3.2-1B and Llama-3.1-8B and cover six unlearning methods. They show that deletion targets are often easy to identify, and that exact-target updates beat all matched controls after 100 steps, sometimes while keeping over 95% of the utility.

Critical strengths
  1. The question is timely. Unlearning is usually evaluated right after the request. The paper rightly points out that in federated settings the model keeps training, so a participating client may try to undo the deletion.

  2. The control design is clean. Every recovery arm starts from the same unlearned state with the same optimizer and number of steps, and only the training data changes. This is a sensible way to separate target-specific recovery from ordinary training.

  3. The evaluation covers six unlearning methods, two model sizes and two forget ratios. It reports recovery together with retained utility, and the recovery-utility frontiers in Appendix E are a useful view.

Critical weaknesses
  1. The most important point, and the one that confuses me most, is that there are no federated experiments. Despite the title and Figure 1, all experiments are run in a centralized setting, with no aggregation, no other clients and no later rounds. The conclusion itself admits that the effect after aggregation remains to be tested. As a result, the main claim about federated unlearning is not supported by the experiments.

  2. The main finding is largely predictable. Fine-tuning on the exact forget records directly optimizes the forget-answer probability, which is the very metric being measured. So it is not surprising that this beats training on retain, holdout or fluent text. Prior relearning work [1] has already shown recovery with much weaker access.

  3. The key control is missing. A model that never saw the forget data could also learn it from the exact records. In Figures 3 to 6, the gold-retrain curves appear to reach a similar level after 100 steps for several methods. Without the retrained model as a control, the results can't tell residual knowledge apart from simply relearning the data.

  4. The threat model is toostrong. The recovery arm receives the exact deletion set as an oracle, separately from the inference step. An attacker who already holds these records could teach them to any model, even a retrained one.

  5. The deletion-target inference is not new, and it doesn't indicate an unlearning failure. Using the pre/post loss change follows Chen et al. (2021). Gold retraining also reaches a ROC-AUC of about 0.999, so high separability is expected for any method that changes the model on the forget set, including exact retraining.

  6. The evaluation is narrow and the statistics are incomplete. Recovery is only tested on TOFU, with a single learning rate and a single unlearned state per method.

  7. Parts of the paper look unfinished? There is a broken reference, and labels such as "Eq. equation 9" appear repeatedly. The appendix figures also use very small fonts and inconsistent styles.

Questions for authors
  1. Could the authors run the attack in an actual federated setting, with aggregation, update clipping and several rounds? Does the recovery survive aggregation?

  2. Could the authors apply the same exact-data updates to the gold-retrained model and use it as a control?

  3. What happens when recovery uses the inferred candidates from RQ1 instead of the oracle set, or uses only paraphrased or partial forget data?

  4. How sensitive are the results to the client numbers, learning rate and the number of steps?

  5. How does this work differ from Sheng's recent work [2], which also studies retaliatory attacks against federated unlearning?

  6. Could the authors test recovery on another benchmark, such as MUSE or WMDP?


[1] Hu, Shengyuan, et al. "Unlearning or obfuscating? jogging the memory of unlearned llms via benign relearning." International Conference on Learning Representations. Vol. 2025. 2025.

[2] Sheng, Xinyi, et al. "Retaliatory attacks against federated unlearning via data leakage." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 30. 2026.

Reviewer_QzHaOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper identifies a recovery risk, where malicious clients can exploit the loss difference between the pre- and post-unlearning states of the global model to spot the forgotten data from their candidate pool. As a result, malicious clients can locally fine-tune on the forgotten data and reintroduce it into the global model in subsequent aggregation rounds.

Critical strengths
  1. The paper identifies an important risk in federated unlearning. Since existing unlearning methods often increase the loss on the data to be forgotten, this could accidentally make it easier for malicious clients to identify the forgotten data by exploiting behavioral differences between pre- and post-unlearning states.

  2. The attack shows strong effectiveness in discerning the forgotten data across different unlearning methods.

Critical weaknesses
  1. The risk that the attacker can exploit the loss difference between model states to identify forgotten data is not necessarily new. In fact, this risk has been identified in previous works (e.g., [1]) and is often referred to as the Streisand effect of unlearning. Additionally, the paper fails to propose a mitigation method for such risk.

  2. The attack success relies heavily on the coverage of the candidate pool (i.e., whether the candidate pool might contain the forgotten data). However, the construction of the candidate pool is largely not discussed in the paper.

  3. The experiment shows that exact-target recovery can restore model behavior on the forgotten data better than in-domain fine-tuning recovery. This empirical finding is rather expected, thus apparently making a marginal contribution to our understanding of the recovery risk.


[1] Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2021). When machine unlearning jeopardizes privacy. Proc. ACM SIGSAC.

Questions for authors
  1. Could the authors elaborate on viable methods that an attacker can use to construct the candidate pool that likely contains the forgotten data? Do you assume that the attacker has prior knowledge about what data is being forgotten?

  2. What are possible defenses to mitigate the unlearning recovery risk identified in this paper?

Reviewer_55LbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the vulnerability of large language models to post-unlearning knowledge recovery in federated unlearning settings. The authors propose Controlled Oracle Recovery, an evaluation framework consisting of two independent components: deletion-target inference and exact-target recovery. The first component identifies potential deletion targets by comparing the prediction loss of candidate records before and after unlearning. The second component fine-tunes the unlearned model using the exact deleted records and compares its recovery performance against updates using retained, holdout, and synthetic data. Experiments are conducted on two Llama model configurations, six unlearning methods, and two forgetting ratios using the TOFU benchmark. The results show that deletion targets can be identified with high accuracy in several settings, while exact-target updates consistently achieve greater recovery than matched control updates. In some settings, the recovered models also maintain high retained utility.

Critical strengths

The paper investigates an important security issue in federated unlearning by examining whether forgotten knowledge can be recovered through subsequent model updates. This perspective extends the evaluation of unlearning beyond its immediate effectiveness and provides useful insights into post-unlearning security risks.

Critical weaknesses

The experimental evaluation is limited in terms of dataset diversity. The main knowledge recovery experiments are conducted only on TOFU-05 and TOFU-10, while WMDP Bio is used solely for supplementary deletion-target inference experiments. Therefore, the generalizability of the reported recovery effects to other types of unlearning tasks remains unclear. Additional recovery experiments on datasets with different forgetting objectives would provide stronger evidence for the broader applicability of the findings.

Questions for authors

In Section 3.3.2 and Algorithm 1, the recovery stage directly uses the exact deleted records rather than the candidates inferred in the preceding stage. How would the recovery performance change if the inferred deletion targets were used instead?

Comments on the paper

1 comment

Thank you for the response and clarifications. I appreciate the additional WMDP experiments and the planned learning-rate analysis. However, I still have a few remaining concerns:

First, coz of lack of aggregation or subsequent federated rounds, the experiment in the paper could only demonstrate local recovery, rather than a successful attack on the global federated model.

Meanwhile, since the evaluation does not distinguish between seen and unseen forget samples, it is difficult to determine whether the observed recovery is from the residual knowledge or relearning from the provided examples.

And for gold-retrained control, thanks for pointing out that this control is included in appx. However, the current comparison seems to have limited statistical support.

Additionally, I did not see the revisions in the updated paper or a more comprehensive discussion addressing these concerns. Based on what mentioned above, I have to keep my original score but lower my confidence to 3. I would also be willing to reconsider my assessment if further experiments could address the issues.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.