- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
PAUSE evaluates whether LLM unlearning can remove requested personal information while preserving information outside the deletion scope. It represents information as subject–attribute–value facts and supports fact-level and subject-level requests. Evaluation separates Target, Same Subject, Co-document, and Global facts. The experiments cover seven corpora, three model families, and eleven unlearning methods. The main finding is that Global preservation scores can obscure damage to related, unrequested information. Additional analyses examine shared values and generation failures.
Critical strengths
-
The four evaluation scopes reveal local preservation failures that a single Global score can miss.
-
Paired fact-level and subject-level requests allow deletion scopes to be compared using the same selected subjects.
-
The document-to-fact mapping and experiments across seven corpora provide a foundation for studying selective PII unlearning.
Critical weaknesses
-
The reported alignment checks do not establish correct ownership, and independent semantic annotation accuracy is not reported. Can the authors provide a human audit and verify the main findings on a checked subset?
-
Relevant PII works (KnowUnDo) already test whether deleting private attributes harms other information about the same person. PAUSE should clarify what its evaluation adds beyond this test and UnlearnPII’s assessment of non-target PII retention.
Questions for authors
-
Existing structured data can already support fact-level and subject-level deletion requests. It remains unclear what additional discovery PAUSE reveals through its document-based construction, beyond grouping facts into different evaluation scopes. Can a controlled comparison using the same facts and requests demonstrate this added diagnostic value?
-
The evaluation omits PII-focused methods such as MemFlex, PPA and PERMU_tok without discussing their applicability. The DPO and RMU adaptations are also underspecified, and a common epoch budget does not establish comparable forgetting strength. Do the preservation gaps persist with a suitably adapted PII baseline at comparable levels of target forgetting?