acceptodds
Under review as a conference paper at ICLR 2027

LLM Unlearning Evaluation with TRIAGE

Abstract

Large language models can memorize private or harmful information, motivating machine unlearning methods that remove targeted knowledge while preserving other capabilities. However, existing evaluations rely primarily on behavioral benchmarks, which assess whether a model appears to forget but provide limited insight into how unlearning changes the model or affects related knowledge. We introduce TRIAGE (Tripartite Representation-internal Introspection for Adjacency Gap Evaluation), a benchmark-agnostic evaluation framework for characterizing these changes. TRIAGE uses diagonal approximations of the Fisher information and Hessian to measure changes in parameter sensitivity and local curvature, and utilizes a Forget / Adjacent-Retain / Generic-Retain partition to quantify an adjacency gap in semantically related knowledge. Based on the magnitude and distribution of these changes, TRIAGE further classifies each algorithm's update as no-op, partially localized, collateral dominant, or globally destructive. Across 12 unlearning methods, four language models, and the WMDP, TOFU, and MUSE benchmarks, we find that methods with similar behavioral forgetting can produce substantially different internal changes and patterns of collateral damage. These signatures also vary across models and benchmarks, indicating that the effects of unlearning are not determined solely by the unlearning algorithm. TRIAGE can be applied alongside existing unlearning benchmarks to complement behavioral evaluation with a model-internal view of how unlearning reshapes the model's parameter space and affects retained knowledge.

Primary area:
datasets and benchmarks
Keywords:
Evaluation Metrics and Protocols, Machine Unlearning, Large Language Models, Societal considerations, Privacy, Alignment

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_shZtOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces TRIAGE, a framework that complements behavioral evaluation of LLM unlearning with model-internal diagnostics. CoFi and CHess use diagonal approximations of the empirical Fisher and Hessian to measure changes in parameter sensitivity and local curvature. A Forget / Adjacent-Retain / Generic-Retain partition helps describe whether these changes are concentrated on the target or extend to retained content. TRIAGE assigns each checkpoint to one of four classes using CoFi. Experiments cover 12 methods, four language models, and WMDP, TOFU, and MUSE, although not every model is tested on every benchmark. The results suggest that similar behavioral forgetting can accompany different internal changes. The paper also includes sensitivity and robustness analyses and a fixed-budget relearning attack.

Critical strengths
  1. The adjacent-retain control is useful. Tables 1 and 2 show that relatively preserved general utility can coexist with large internal sensitivity changes on semantically adjacent retained content. Bringing this control into model-internal evaluation adds useful information beyond a standard Forget / Generic-Retain comparison.

  2. The experimental coverage is broad, and the interpretation is mostly careful. The paper examines several methods, models, and benchmarks, with additional checks on sample size, probe count, and diagnostic subspace. The authors explicitly acknowledge that internal change does not establish knowledge removal. This makes the results useful as complementary diagnostics rather than a certificate of successful unlearning.

Critical weaknesses
  1. The diagnostic interpretation of AdjGap needs stronger validation. The authors acknowledge that structural change does not imply behavioral damage. Still, disagreement between structural and behavioral signals suggests non-redundancy, but does not by itself validate AdjGap as an indicator of collateral damage. The relearning results mainly connect CoFi changes with recovery; they do not clearly establish the incremental diagnostic value of AdjGap beyond the overall magnitude of CoFi changes. Labels such as “collateral dominant” may suggest that actual damage has been demonstrated.

  2. Uncertainty around class boundaries remains important. The authors provide some stability analyses and acknowledge uncertain boundary cases, but classification still mainly follows point estimates. For example, RMU's AdjGap in Table 1 is only 1.01 percentage points. A confidence interval for AdjGap itself, accounting for the joint uncertainty of its components, would help determine whether the sign used for classification is reliable. The separate intervals for Forget and Adjacent-Retain do not answer this directly.

Questions for authors
  1. After accounting for initial behavioral forgetting and general utility, does AdjGap add information about relearning recovery or damage on Adjacent-Retain content?
  2. Which class assignments remain reliable when using paired confidence intervals for AdjGap and repeated unlearning runs with different training seeds?
2 comments
AuthorsOpenReview ↗

Polite ping

Dear reviewer,

We believe we have addressed all concerns raised in your review. Please let us know if there are any outstanding issues you would like us to address. We would sincerely appreciate your engagement and time for engagement is running out.

Again, thanks a lot for your valuable feedback.

Sincerely,

The authors

Most of my concern has been addressed, and I have raised my score.

Reviewer_mufZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TRIAGE, a framework for evaluating internal changes caused by LLM unlearning. CoFi and CHess measure changes in parameter sensitivity and local curvature using diagonal empirical Fisher and Hessian estimates. A Forget / Adjacent-Retain / Generic-Retain partition examines how these changes extend beyond the target knowledge. CoFi determines four update classes; CHess and perplexity provide auxiliary diagnostics. Experiments cover 12 methods, four models, and WMDP, TOFU, and MUSE, with some model-benchmark combinations omitted. The results show that structural signatures vary across methods, models, and benchmarks, including cases with similar behavioral forgetting.

Critical strengths
  1. Combining adjacent-retain controls with internal diagnostics: Adjacent-retain evaluation has precedents, as the paper acknowledges. The useful contribution here is to incorporate this control into measurements of parameter sensitivity and curvature. This distinguishes structural changes on semantically related content from those on generic retained content, allowing a more detailed description of the unlearning update. The framework is presented as a complement to behavioral evaluation, rather than evidence that knowledge has been removed.

  2. Comparisons across methods, models, and benchmarks: The experiments show how the same method can produce different structural signatures under different conditions. These results help define both the scope and the limits of the proposed diagnostic.

Critical weaknesses
  1. The information added beyond behavioral measures is not sufficient: In the Llama-3.1-8B WMDP results, RMU and Adaptive-RMU initially lose approximately 0.2760 and 0.3075 in mean Bio/Cyber accuracy, while relearning restores 0.2738 and 0.3020, respectively (Tables 36 and 39). Recovery is therefore close to the initial loss in these cases. This raises the question of how much the association in Section 4.6 reflects the amount previously forgotten and the model's retained utility. Table 1 already shows substantial adjacent-retain perplexity increases for several methods, so behavioral evaluation is detecting changes beyond accuracy alone. The central missing analysis is whether CoFi adds information about relearning response or collateral effects beyond these existing behavioral measures. The current correlations do not resolve that question.

  2. Classification uncertainty and corpus coverage limit some interpretations: Adaptive-RMU's AdjGap is -0.19 percentage points and RMU's is +1.01 in Table 2. Intervals for individual corpus measurements do not establish uncertainty in their difference or in the resulting class. Appendix C's local checks do not settle the stability of these boundary cases under a fixed model and benchmark. For MUSE-Books, Appendix F.3 states that the structural estimate uses the first 200 windows, all from the first of four forget books. This limits its coverage of the deletion target. The comparison with WMDP therefore needs a coverage check before its structural contrast can be confidently interpreted at the benchmark level.

Questions for authors
  1. Added information in the relearning analysis: Using the existing checkpoints, can you assess whether CoFi contributes information about recovery after accounting for initial forgetting and retained utility, including accuracy and perplexity?

  2. Uncertainty in class assignments: Can you quantify uncertainty in AdjGap and the globality ratio by resampling within each corpus, while preserving the pairing between base-model and unlearned-model evaluations on the same samples?

  3. MUSE-Books sampling: With the same total window budget, does sampling across all four forget books and different positions within each book preserve the reported contrast with WMDP? This would directly test the effect of corpus coverage without requiring a larger evaluation budget.

4 comments

Dear reviewer,

We believe we have addressed all concerns raised in your review. Please let us know if there are any outstanding issues you would like us to address. We would sincerely appreciate your engagement and time for engagement is running out.

Again, thanks a lot for your valuable feedback.

Sincerely,

The authors

Thank you for the detailed response and additional analyses. These results substantially address my main concerns. The partial-correlation analysis supports a consistent association between CoFi and recovery. The paired resampling also clarifies the classification uncertainty. Stratified sampling across all four books addresses my concern about corpus coverage. The paper should still address the following points in the revision. First, the conditional association with recovery should be distinguished from predictive performance, since no improvement in leave-one-model-out recovery prediction has been demonstrated. The limited discrimination within the active, non-collapsed subset should also be noted. Second, what evaluation budget is intended for the revised MUSE-Books protocol, given the 600 windows in the response and the original 200-window description? How does sampling across all four books differ from evaluating the entire corpus in terms of coverage and diagnostic purpose? Once these analyses and methodological details are incorporated into the paper, I believe the evidence would support TRIAGE as a useful diagnostic tool, and I would be happy to raise my rating.

We thank the reviewer for the thoughtful follow-up and for the willingness to raise the rating. We will incorporate both points in the revision.

1. Conditional association vs. predictive performance. We agree, and will make this distinction explicit in Section 4.6 and the appendix.

  • What we show: a conditional association between ΔCoFi and relearning recovery across all checkpoints, after controlling for initial forgetting, retained utility and perplexity (partial ρ = 0.43).
  • What we do not show: improved recovery prediction. Adding ΔCoFi to forgetting and utility did not reduce leave-one-out prediction error. (Our predictive check left out one checkpoint at a time rather than one model; under neither scheme do we claim a predictive improvement.)
  • The active subset: we will note that among the 16 active, non-collapsed checkpoints, the fixed-budget attack restores a median 96% of initial forgetting, which limits its ability to discriminate within that subset. The subset analysis will be reported in the appendix.

2. MUSE-Books evaluation budget. The revised protocol keeps the original compute budget but spends it on distinct windows from all four books:

Forget set, per checkpointOriginal protocolRevised protocol
Window evaluations3 subsets × 200 = 60010 subsets × 60 = 600
Distinct windows200 (the three subsets coincide)600
Books covered1 of 4 (first 200 windows of book 1)4 of 4, in proportion to book length
Positions within each bookbeginning of book 1 only5 position bins per book
Share of the forget corpus (952 windows)21%63%
Uncertainty estimatenonebootstrap over 10 disjoint subsets

The 200-window figure in the paper was the size of each subset, and on MUSE-Books all three subsets contained the same windows. The 600 in our response is the same total of window evaluations, now spread over distinct windows. For the retain splits, we use as many disjoint 60-window subsets as each split contains (3 for retain1, 7 for retain2).

Four-book sampling vs. the entire corpus.

  • Full corpus: evaluating all 952 windows would remove sampling error from the forget-set estimate, at about 1.6× the cost. But it yields a single value with no within-corpus uncertainty estimate.
  • Stratified disjoint subsets: these give an estimate of the full-corpus value that represents every book and every part of each book. They also provide the intervals needed to judge whether a class assignment is stable, which is the diagnostic purpose of TRIAGE.
  • Comparability: a fixed budget keeps the cost and precision comparable across benchmarks of very different sizes.
  • Effect on classes: the resulting intervals are narrow relative to the class boundaries. For example, NPO's forget shift is 13.19% [11.78, 15.11], against 0.02% on adjacent content. Every class is far from its decision boundary.

We will state the budget, the coverage fraction and this rationale in Appendix F.3.

Thank you for the clarification. Both points are now clear. Together with the earlier analyses, these explanations address my main concerns.

Reviewer_UtwBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduces TRIAGE (Tripartite Representation-internal Introspection for Adjacency Gap Evaluation), a method that aims to evaluate the effects of LLM unlearning. The idea behind is to test not only model's asnwers but also how the internal parameters respond to different categories of information (forget, adjacent-retain, generic-retain). The use two measures that rely on Fischer informtion and Hessian. In simple terms this study asks the following. When we try to make the model forget something, how do we know that it actually forgot it and didn't reduce/damaged other knowledge?

Critical strengths
  1. Important and difficult problem. We need more robust methods to evaluate forgetting because (strangely) forgetting in this context doesn't mean that the information has been removed.

  2. There is a solid amount of evaluations present in the paper. This is done across methods, models and benchmarks and interesting differences highlighted in the paper.

Critical weaknesses

Probably the main weakness is related with validity of the structural metrics. What do changes in Fischer information and Hessian really tell us? This is despite some efforts to conduct some relearning experiments.

This is not a critical weakness but the Appendix would benefit from some additional work and attention. It feels like things were included there without much thought or organisation, which lowers the quality of the exposition.

Questions for authors

1.The authors claim (sec 3.2) that a large CoFi (Concept Fischer) shift indicates structure change, but does it mean that the knowledge has been removed? 2. The authors claim that the 4 categories (triage classes, sec 3.6) are determined by the CoFi alone. How reliable are these classes? Would the classification change if we were to consider Hessian and perplexity?

1 comment

Dear reviewer,

We believe we have addressed all concerns raised in your review. Please let us know if there are any outstanding issues you would like us to address. We would sincerely appreciate your engagement and time for engagement is running out.

Again, thanks a lot for your valuable feedback.

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.