acceptodds
Under review as a conference paper at ICLR 2027

UnlearnBench: A Clean and Controllable Benchmark for Fine-Grained and Compositional LLM Unlearning

10 views

Abstract

LLM unlearning is essential for ensuring technical and responsible AI use, especially in addressing privacy leakage and evolving regulatory requirements. Existing unlearning benchmarks primarily evaluate unlearning through shallow question answering suppression, while overlooking several fundamental challenges: Knowledge in LLMs is acquired through complex memorization processes. It is further organized in compositional, multi-stage, and structurally entangled forms. As a result, it remains unclear whether current unlearning methods can reliably remove knowledge learned through staged training pipelines, relational reasoning, and fine-grained attribute associations. In this work, we introduce UnlearnBench, a benchmark for studying unlearning across diverse settings including single-hop QA, two-stage memorization, multi-hop reasoning, and fine-grained attribute-level forgetting. Compared with existing benchmarks, UnlearnBench better captures realistic memorization and retrieval behaviors while enabling systematic analysis of unlearning performance under clean and controlled settings. We additionally provide clean pretrained models and exact retraining checkpoints as gold-standard references for future evaluation. Extensive experiments on popular unlearning methods uncover several consistent failure modes. First, entity-level unlearning often fails even in settings where retraining successfully removes the target knowledge. Second, reasoning abilities can recover supposedly forgotten information from retained representations, revealing strong interactions between memorization and compositional reasoning. Third, attribute-level knowledge is highly entangled across entities and relation types, making selective forgetting substantially more difficult than existing benchmarks suggest. Our results demonstrate that current evaluations significantly underestimate the challenges of LLM unlearning and establish UnlearnBench as a structured testbed for future unlearning research.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Unlearning, Unlearning Evaluation, LLM Memorization

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_D2StOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce UnlearnBench to address the limited coverage of compositional reasoning, multi stage memorization, and attribute level forgetting in previous benchmarks, which mainly focus on single hop QA and entity or document level unlearning. The benchmark allows precise control over knowledge provenance, memorization stages, and forget and retain splits. Experiments evaluate several unlearning methods and identify different failure modes across settings.

Critical strengths

Overall, the paper is easy to follow. It provides a useful benchmark for studying selective forgetting of attributes and unlearning in multi hop reasoning settings.

Critical weaknesses
  1. The paper could be clearer with more figures and fewer typos.
  2. The experiments need clearer training details and tests on more models.
Questions for authors
  1. The benchmark design could be explained more clearly with an overview figure showing how UnlearnBench differs from existing benchmarks. Figures summarizing the main experimental findings would also help readers follow the analysis.
  2. There are several typos and formatting issues, including:
    1. “allen-zhu2023storage” in L155–156.
    2. “qwen3technicalreport” in L161.
    3. “BLUR (?)” in Table 2.
    4. The number of methods stated in L262 appears inconsistent with the methods listed.
  3. Many experimental results fall into two extremes: the model loses both target and retained knowledge, or it barely forgets the target knowledge. This makes the methods difficult to compare and raises the question of whether the results are due to limitations of the methods or unsuitable training settings. Although Appendix D.7 includes some results with different learning rates and training steps, the tuning budget, search ranges, and checkpoint selection criteria remain unclear.
  4. Most experiments focus on Qwen3 0.6B. Although the appendix includes additional Llama results, the main findings on compositional reasoning and multiple training stages are not sufficiently tested across models. Evaluating these settings on another model would help establish whether the findings hold more broadly.
Reviewer_EjdcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces UnlearnBench, a controllable benchmark for evaluating LLM unlearning beyond single-hop QA. It provides a synthetic dataset of people with biographies and relationships, uses multi-stage training, and evaluates entity-level, attribute-level, and two-hop unlearning. The experiments show that selective forgetting can be much harder than what prior benchmarks suggest, especially when knowledge is learned across multiple training stages or can still be recovered through reasoning.

Critical strengths
  1. The benchmark includes several multi-stage memorization settings, such as pretraining, QA fine-tuning, and two-hop reasoning. This makes it possible to study how knowledge storage and knowledge extraction affect unlearning.

  2. It includes both entity-level and attribute-level unlearning, which allows more fine-grained evaluation.

  3. The release of retrained checkpoints provides a useful reference for what successful forgetting should look like.

Critical weaknesses
  1. The benchmark might contain many people with very similar biographies. Many biography and QA samples may also be written in similar ways. This could make the knowledge of different people more entangled and make unlearning harder than in real-world cases. The paper measures diversity between different biographies of the same person, but does not clearly measure similarity between different people.

  2. Utility outside the benchmark is not considered. After training the target model, most of the knowledge and utility being evaluated come from this synthetic dataset. Depending on the training setup, the model might also forget some or a large amount of its previous knowledge, but the paper does not show how much useful ability the target model still has outside the benchmark. This can affect how the unlearning results should be interpreted. For example, forgetting 1% or 10% of this dataset may mean removing a relatively large part of the knowledge that the target model currently relies on.

  3. TOFU is already a synthetic and controlled dataset where the source of the knowledge is known. This seems inconsistent with the paper saying that benchmarks such as TOFU use uncontrolled corpora and training settings.

  4. The paper often uses the phrase “training from scratch,” but the main experiments start from pretrained HuggingFace base models and then continue pretraining or fine-tuning. Training from random initialization and continuing training from a pretrained model are very different settings.

  5. Around line 156 and in Table 2, there are citation errors.

  6. The paper mainly focuses on two small LLMs, Qwen3-0.6B and Llama-3.2-1B. Model size can affect how knowledge is stored and how unlearning methods perform, so the results may not directly transfer to larger models.

  7. Some closely related works, such as The Unlearning Mirage, are not discussed, even though they also show that supposedly forgotten information can still be recovered through reasoning. Also, based on the paper's own MUSE experiments, current unlearning methods already show the same basic failure on real-world data: they either fail to forget or damage both forget and retain knowledge. This makes it less clear which failure modes are actually new to UnlearnBench.

Shah, Raj Sanjay, et al. "The unlearning mirage: A dynamic framework for evaluating LLM unlearning." arXiv preprint arXiv:2603.11266 (2026).

  1. The hyperparameter selection for the unlearning baselines is not clear. The results can change a lot depending on the number of optimization steps, and some methods suddenly go from almost no forgetting to complete collapse. Because of this, the paper should explain more clearly how the hyperparameters for each method were selected.
Questions for authors
  1. How were the hyperparameters and number of optimization steps selected for each unlearning method? Since some methods suddenly go from almost no forgetting to complete collapse, were all methods tuned using the same selection criterion and a similar tuning budget?

  2. Previous work such as The Unlearning Mirage already shows that supposedly forgotten information can be recovered through reasoning, and the paper's own MUSE experiments show that unlearning methods either fail to forget or damage retain knowledge. Which failure modes are specifically new to UnlearnBench compared with these previous results?

  3. After training the target model on the synthetic dataset, how much of its original general knowledge and abilities are still preserved? Since utility is mainly evaluated within the benchmark, could forgetting 1% or 10% of this dataset represent a much larger part of the useful knowledge the target model relies on than it would in a more realistic unlearning setting?

Reviewer_CnBMOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes UnlearnBench, a synthetic LLM unlearning benchmark that consists of biographical text and question-answer pairs regarding 100k fictional individuals. Each individual is accompanied by six attributes (birth date, birth city, university, major, employer, and work city) and four relations (mentor, colleague, advisor, neighbor) to another individual, together enabling compositional question answering. The paper tests various ways knowledge can be injected to the model: (1) mixed training on biography and QA data, (2) biography pretraining and QA finetuning, and (3) a three-stage CPT pipeline (Bio, QA, then chain-of-thought QA), each accompanied by a clear counterfactual retrain-from-scratch checkpoint. Evaluating existing unlearning frameworks on entity-, attribute-, and relation-level unlearning under accuracy and likelihood-based metrics show that all consistently fail in unlearning and are unable to approach the retain-only model.

Critical strengths
  • [S1] Targets pretrained knowledge with ground-truth counterfactual models. Most unlearning benchmarks simulate an unlearning scenario through finetuning already pretrained models, which are not exactly aligned with real-world scenarios where unlearning would be most relevant. Many also do not provide the retrain-from-scratch model, which makes it difficult to assess unlearning efficacy accurately. This paper is likely the first benchmark that not only targets pretraining data, but also provides the gold-standard model for each training scenario.
  • [S2] Interesting analysis on source of knowledge. Separating how a model can acquire knowledge of particular individuals (Bio vs. QA) is an interesting take, and proposes a new view on how unlearning should tackle the two steps (storage and extraction) separately.
Critical weaknesses
  • [W1] Limited novelty. The synthetic-person pretraining setup and the mixed training on Bio and QA data are largely borrowed from the Physics of LM line of work [A]. Evaluating beyond shallow single-hop QA recall was also considered in previous work on knowledge graph unlearning, an example being [B]. The main novelty thus lies in the fact that we are able to cleanly target pretraining data and evaluate with clear counterfactuals, but even so, the contribution seems a bit shallow (more on next point W2).
  • [W2] Shallow evaluation despite access to retrained models. The main results hinge on QA accuracies and likelihoods, and while the Appendix shares results using the MIA metric [C], it is limited to just one setup. Since the paper already has retrained checkpoints, findings from comparing the unlearned model to those gold-standard checkpoints using metrics such as forget quality [D] and distributional equivalence [E] would be much more interesting. Without these, the main results are still very much limited to the "shallow QA suppression" that the paper is claiming to improve upon.
  • [W3] Questionable generalizability of key findings. The highly structured synthetic data makes analysis clean, but at the same time greatly limits the generalizability of the results. For example, the finding that knowledge is organized by attribute type may be an artifact of this template-driven pretraining data. Any discussion on what findings the authors believe would be transferred into less structured data would be useful.

[A] Allen-Zhu and Li. Physics of Language Models: Part 3.1, Knowledge Storage and Extraction. arXiv 2023.
[B] Wei et al., Do LLMs Really Forget? Evaluating Unlearning with Knowledge Correlation and Confidence Awareness. NeurIPS 2025
[C] Shi et al., MUSE: Machine Unlearning Six-Way Evaluation for Language Models. ICLR 2025.
[D] Maini et al., TOFU: A Task of Fictitious Unlearning for LLMs. COLM 2024.
[E] Cho et al., Reference-Specific Unlearning Metrics Can Hide the Truth: A Reality Check. arXiv 2025.

Questions for authors
  • [Q1] In the two-hop setups, it appears that the "QA-only" forget sets contains only the QA dataset and not the biography datasets, and as a result even the retrain-from-scratch models show high accuracy on QAforgetQA_{\text{forget}} (e.g., Retrain (S2+QA) row in Table 5). In such cases, is it correct to state that "forgetting is difficult" when the oracle model itself shows high accuracy? This seems more like an ill-posed problem where the forget set is not comprehensive enough to represent the knowledge we wish to unlearn.
  • [Q2] The unlearning setups could be listed a bit more clearly. The draft is a bit hard to follow, particularly because each unlearning setting is described in plain text, without explicitly stating what the forget set DfD_f and retain set DrD_r are. A table describing (1) what datasets the starting base model was trained on, (2) what data points we are hoping to remove, and (3) what retain data the unlearning algorithm can use for each unlearning setup would greatly improve readability.
  • [Q3] Just a minor issue on citations: "allen-zhu2023storage" in Line 155, "qwen3technicalreport" in Line 161, "BLUR (?)" in Table 2.
Reviewer_JFXeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper creates a benchmark with synthetic people, their biographies and questions about them that would be used to evaluate the unlearning methods in a structured way depending on the request. The request are differing such as unlearning all information about a person, a specific question/query about that person, or relationships between individuals.

The authors tested different LLM unlearning methods under this benchmark and provide intuitive explanations. They have stated that the existing methods often struggle to achieve unlearning performance intended.

Critical strengths
  • [S1] I think it is important to have a generally accepted dataset benchmark on unlearning/privacy to evaluate heuristic unlearning methods (especially in the context of generative models) and this paper targets this purpose.

  • [S2] The construction of the dataset makes sense in general from the privacy perspective since information is directly about specific individuals. Also, having relations between individuals makes the datasets more realistic and applicable for real life.

Critical weaknesses
  • [W1] Because the constructed datasets contains relations, making the answer disappear in the generation does not necessarily mean the unlearned model is close to the retrained from scratch model. Also, the training conducted on retrained model is also important since it might be overfitted or not generalizable enough therefore specifying limitations on the target is also important. The main score to evaluate the methods increases when the accuracy on the unlearned samples decreases however this evaluation metric might be misleading. The paper also discusses the membership inference attack which can be seen as a more proper way to evaluate unlearning however the main claims mainly depends on the accuracy on different data splits. Finally, the attack proposed also check the distinguishability between the retained samples and forget samples however an accurate way of approaching is to check the distinguishability between the target model and the unlearned model outputs on different data samples.

  • [W2] The diversity in the data samples is also important if the whole evaluation mainly focus on the biographies and the biographies are having similar structures then that would not be the evaluation dataset reconstruction. As far as I understand the construction mainly focus on 6 features on an individual, I am unsure on how representative is this.

Questions for authors
  • [Q1] If we have a highly connected relations between individuals, how we can evaluate unlearning when a person's entire identity is requested to be deleted?

  • [Q2] How we can set the limits for the retrained model for highly complex models so that the target itself would be also explainable?

  • [Q3] Do you think the current construction of the datasets captures most of the real world use cases? What could be the limitations or next steps on this construction?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.